May 07 15:25:22.291561 np0000172501 systemd[1]: Started devstack@neutron-ovn-maintenance-worker.service - Devstack devstack@neutron-ovn-maintenance-worker.service.
May 07 15:25:22.624548 np0000172501 neutron-ovn-maintenance-worker[104723]: /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:41: EventletDeprecationWarning:
May 07 15:25:22.624548 np0000172501 neutron-ovn-maintenance-worker[104723]: Eventlet is deprecated. It is currently being maintained in bugfix mode, and
May 07 15:25:22.624548 np0000172501 neutron-ovn-maintenance-worker[104723]: we strongly recommend against using it for new projects.
May 07 15:25:22.624548 np0000172501 neutron-ovn-maintenance-worker[104723]: If you are already using Eventlet, we recommend migrating to a different
May 07 15:25:22.624548 np0000172501 neutron-ovn-maintenance-worker[104723]: framework.  For more detail see
May 07 15:25:22.624548 np0000172501 neutron-ovn-maintenance-worker[104723]: https://eventlet.readthedocs.io/en/latest/asyncio/migration.html
May 07 15:25:22.624548 np0000172501 neutron-ovn-maintenance-worker[104723]:   import eventlet
May 07 15:25:23.637018 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.common.config [-] Logging enabled!
May 07 15:25:23.637660 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker version 28.1.0.dev195
May 07 15:25:23.637660 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=104723) setup_logging /opt/stack/neutron/neutron/common/config.py:124}}
May 07 15:25:23.637660 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.server.ovn_maintenance [-] OVN maintenance process starting...
May 07 15:25:23.637660 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "plugin-directory" by "neutron_lib.plugins.directory._create_plugin_directory" {{(pid=104723) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
May 07 15:25:23.638373 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=104723) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
May 07 15:25:23.638373 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" "released" by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.000s {{(pid=104723) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
May 07 15:25:23.638373 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "manager" by "neutron.manager.NeutronManager._create_instance" {{(pid=104723) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
May 07 15:25:23.638373 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=104723) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
May 07 15:25:23.638373 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.manager [-] Loading core plugin: ml2
May 07 15:25:23.907099 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:23.907099 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:23.907453 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:23.907838 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:23.907930 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:23.907930 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:23.942987 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:23.943169 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:23.943169 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.038066 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.038309 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.038309 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.056913 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.057065 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.057065 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.057611 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.057768 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.057768 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.141016 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x7bc779f825c0> agent after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.141286 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x7bc779f825c0> agent after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.141286 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _add_segment_host_mapping_for_segment at 0x7bc779f82660> segment precommit_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.141286 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _delete_segments_for_network at 0x7bc779f82700> network precommit_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.141577 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _prevent_segment_delete_with_port_bound at 0x7bc779f83920> segment before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.190912 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._before_network_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> network before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191111 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function DVRDbMixin._delete_mac_associated_with_agent at 0x7bc779f72020> agent before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191111 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> network before_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191111 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> port before_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191429 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> port before_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191429 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> segment precommit_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191429 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> segment precommit_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191429 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> segment after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191429 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> segment after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191649 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> network after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191649 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> network precommit_delete 0, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191649 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._port_provisioned of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> port provisioning_complete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191778 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_create of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> rbac-policy before_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191778 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_delete of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> rbac-policy after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191778 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._retry_binding_revived_agents of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> agent after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191896 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> subnet after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191896 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> subnet precommit_delete 0, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.191979 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> rbac-policy before_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.192027 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> rbac-policy before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.192076 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> segment_host_mapping after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.192141 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin.delete_agent_notified of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> agent after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.192278 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> rbac-policy before_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.192341 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> rbac-policy before_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.192401 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7bc779ebb290>> rbac-policy before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.192708 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=104723) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 07 15:25:24.192775 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=104723) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 07 15:25:24.192824 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=104723) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 07 15:25:24.192875 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=104723) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 07 15:25:24.192928 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=104723) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 07 15:25:24.192974 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=104723) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 07 15:25:24.193141 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'geneve']
May 07 15:25:24.193858 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['public']
May 07 15:25:24.194427 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
May 07 15:25:24.194900 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict({'public': [(1, 4094)]})
May 07 15:25:24.194965 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'local', 'vlan']
May 07 15:25:24.195154 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'local', 'vlan'])
May 07 15:25:24.195239 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [-] Project network_types: ['geneve']
May 07 15:25:24.195346 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security', 'qos']
May 07 15:25:24.195831 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security', 'qos']
May 07 15:25:24.195886 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security', 'qos']
May 07 15:25:24.195974 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn']
May 07 15:25:24.232182 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn']
May 07 15:25:24.232182 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn']
May 07 15:25:24.232447 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling.
May 07 15:25:24.233024 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x7bc779a3cd40>> floatingip before_response 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.233139 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x7bc779a3cd40>> port before_response 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:24.251923 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat'
May 07 15:25:24.251923 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete
May 07 15:25:24.252115 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve'
May 07 15:25:24.252115 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 65536)]
May 07 15:25:24.297016 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO dbcounter [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Registered counter for database neutron
May 07 15:25:24.309552 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG oslo_db.sqlalchemy.engines [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=104723) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:325}}
May 07 15:25:24.708032 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG dbcounter [-] [104723] Writer thread running {{(pid=104723) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
May 07 15:25:24.894763 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Initializing driver for type 'local'
May 07 15:25:24.894956 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Initializing driver for type 'vlan'
May 07 15:25:25.004736 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] VlanTypeDriver initialization complete
May 07 15:25:25.004868 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Initializing extension driver 'port_security'
May 07 15:25:25.004941 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.extensions.port_security [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] PortSecurityExtensionDriver initialization complete
May 07 15:25:25.004941 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Initializing extension driver 'qos'
May 07 15:25:25.005066 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.plugins.ml2.extensions.qos [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] QosExtensionDriver initialization complete {{(pid=104723) initialize /opt/stack/neutron/neutron/plugins/ml2/extensions/qos.py:31}}
May 07 15:25:25.005125 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Initializing mechanism driver 'ovn'
May 07 15:25:25.005191 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Starting OVNMechanismDriver
May 07 15:25:25.006572 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver.pre_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> process before_spawn 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.006681 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver.post_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> process after_init 55550000, True {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.006750 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver._add_segment_host_mapping_for_segment of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> segment after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.006850 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver.create_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> segment after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.006913 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver.delete_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> segment after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.007017 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver._validate_allowed_address_pairs of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> allowed_address_pair before_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.007084 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver._create_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> security_group precommit_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.007155 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver._update_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> security_group after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.007259 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver._create_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> security_group after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.007339 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> security_group precommit_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.007420 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> security_group after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.007580 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver._create_sg_rule_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> security_group_rule precommit_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.007708 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> security_group_rule after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.007708 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> security_group_rule before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.007845 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> address_group after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.007845 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> address_group after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.008120 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7bc779abad50>> address_group after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.008120 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method DriverBase._register of <neutron.services.qos.drivers.ovn.driver.OVNQosDriver object at 0x7bc77993b0e0>> qos_plugin after_init 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.008275 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNTrunkDriver.register of <neutron.services.trunk.drivers.ovn.trunk_driver.OVNTrunkDriver object at 0x7bc779a6c0e0>> trunk_plugin after_init 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.008436 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method DriverBase._register of <neutron.services.logapi.drivers.ovn.driver.OVNDriver object at 0x7bc778ce9790>> logging-plugin after_init 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.008790 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.services.logapi.drivers.ovn.driver [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] OVN logging driver registered
May 07 15:25:25.009969 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.plugins.ml2.plugin [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=104723) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
May 07 15:25:25.010178 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc77957c7d0>> port after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.010290 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc77957c7d0>> port after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.010362 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc77957c7d0>> port after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.010433 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce9a00>> subnet after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.010495 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce9a00>> subnet after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.010572 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce9a00>> subnet after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.010682 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce9970>> network after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.010758 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce9970>> network after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.010828 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce9970>> network after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.010899 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce9910>> security_group after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.010960 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce9910>> security_group after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.011032 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce9910>> security_group after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.011101 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce98e0>> security_group_rule after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.011171 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce98e0>> security_group_rule after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.011292 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce98e0>> security_group_rule after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.011368 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce9940>> address_group after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.011439 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce9940>> address_group after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.011549 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7bc778ce9940>> address_group after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.011620 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ML2 OVO RPC backend initialized. {{(pid=104723) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:117}}
May 07 15:25:25.012264 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.quota [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded quota_driver: <neutron.db.quota.driver_nolock.DbQuotaNoLockDriver object at 0x7bc778ce9880>.
May 07 15:25:25.012332 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.plugin [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Modular L2 Plugin initialization complete
May 07 15:25:25.012456 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading services supported by the core plugin {{(pid=104723) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:170}}
May 07 15:25:25.012536 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.plugins.ml2.managers [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Got port-security extension from driver 'port_security'
May 07 15:25:25.012773 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading service plugins: ['ovn-router', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=104723) _load_service_plugins /opt/stack/neutron/neutron/manager.py:195}}
May 07 15:25:25.012853 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading Plugin: ovn-router
May 07 15:25:25.019776 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._delete_router_remove_external_gateways of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7bc778ceb110>> router before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.019895 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <function L3RpcNotifierMixin._notify_gateway_port_ip_changed at 0x7bc779bcfe20> port after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.019982 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <function L3RpcNotifierMixin._notify_routers_callback at 0x7bc779bcff60> port after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.020058 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <function L3RpcNotifierMixin._notify_subnet_gateway_ip_update at 0x7bc779bcfec0> subnet after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.020143 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <function L3RpcNotifierMixin._notify_subnetpool_address_scope_update at 0x7bc779bcfd80> subnetpool_address_scope after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.020254 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x7bc778d2b7e0> port before_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.020328 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x7bc778d2b7e0> port after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.020411 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method L3_NAT_dbonly_mixin._precommit_delete_port_callback of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7bc778ceb110>> port precommit_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.020492 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_internal_ip_change_for_fip at 0x7bc779bc7600> port before_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.020570 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback at 0x7bc779bc7560> port before_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.020650 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method RouterAvailabilityZoneMixin._process_az_request of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7bc778ceb110>> router precommit_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.020732 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_create of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7bc778ceb110>> router precommit_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.020811 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_update of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7bc778ceb110>> router precommit_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.020897 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <function OVNL3RouterPlugin._subnet_update at 0x7bc778d2b740> subnet after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.021493 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.quota.resource_registry [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Tracking information for resource: router configured {{(pid=104723) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 07 15:25:25.021548 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.quota.resource_registry [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Tracking information for resource: floatingip configured {{(pid=104723) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 07 15:25:25.021637 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.services.ovn_l3.plugin [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Starting OVNL3RouterPlugin
May 07 15:25:25.021808 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNPortForwarding.register of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7bc778ceb650>> port_forwarding_plugin after_init 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.021971 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method DriverController._check_router_request of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7bc778ceb020>> router before_create 55550100, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.022052 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method DriverController._clear_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7bc778ceb020>> router precommit_delete 55550100, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.022198 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method DriverController._set_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7bc778ceb020>> router precommit_create 55550100, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.022259 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method DriverController._update_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7bc778ceb020>> router precommit_update 55550100, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.024603 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.services.provider_configuration [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Service providers = [] {{(pid=104723) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:170}}
May 07 15:25:25.024670 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.db.servicetype_db [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=104723) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:45}}
May 07 15:25:25.025721 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVN_L3_HA_db_mixin._precommit_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> router precommit_create 55549900, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.025802 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OvnDriver._process_add_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> router_interface after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.025886 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OvnDriver._process_floatingip_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> floatingip after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.025973 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> floatingip precommit_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.026033 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> floatingip precommit_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.026127 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OvnDriver._process_floatingip_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> floatingip after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.026193 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OvnDriver._process_floatingip_status_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> floatingip after_status_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.026286 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OvnDriver._process_floatingip_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> floatingip after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.026356 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OvnDriver._process_portforwarding_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> port_forwarding after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.026491 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OvnDriver._process_remove_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> router_interface after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.026675 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OvnDriver._process_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> router after_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.026675 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OvnDriver._process_router_create_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> router precommit_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.026675 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OvnDriver._process_router_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> router after_delete 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.026855 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OvnDriver._process_router_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7bc778d1bf80>> router after_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.026855 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.services.service_base [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded 'neutron.services.ovn_l3.service_providers.ovn.OvnDriver' provider for service L3_ROUTER_NAT {{(pid=104723) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}}
May 07 15:25:25.026980 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <bound method OVNL3RouterPlugin._post_fork_initialize of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7bc778ceb110>> process after_init 55550001, True {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.027040 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding using OVN {{(pid=104723) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 07 15:25:25.027104 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading Plugin: auto_allocate
May 07 15:25:25.027850 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x7bc778d3da80> network precommit_update 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.027933 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron_lib.callbacks.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x7bc778d3da80> network precommit_create 55550000, False {{(pid=104723) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.028073 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=104723) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 07 15:25:25.028144 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading Plugin: tag
May 07 15:25:25.030434 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=104723) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 07 15:25:25.030506 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading Plugin: timestamp
May 07 15:25:25.031433 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=104723) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 07 15:25:25.031499 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading Plugin: network_ip_availability
May 07 15:25:25.032684 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=104723) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 07 15:25:25.032750 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading Plugin: flavors
May 07 15:25:25.033493 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=104723) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 07 15:25:25.033579 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading Plugin: revisions
May 07 15:25:25.034341 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.manager [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=104723) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 07 15:25:25.034433 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG oslo_concurrency.lockutils [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Lock "manager" "released" by "neutron.manager.NeutronManager._create_instance" :: held 1.396s {{(pid=104723) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
May 07 15:25:25.034604 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] get_extension_paths = OrderedDict({'/opt/stack/neutron/neutron/extensions': 1}) {{(pid=104723) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:640}}
May 07 15:25:25.034676 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Initializing extension manager.
May 07 15:25:25.034894 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: __init__.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.034971 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: __pycache__ {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.035034 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: address_group.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.035362 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.035577 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: address-group
May 07 15:25:25.035651 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: address_scope.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.035892 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.036041 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: address-scope
May 07 15:25:25.036105 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: admin_state_down_before_update.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.036329 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.036460 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension router-admin-state-down-before-update not supported by any of loaded plugins
May 07 15:25:25.036516 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: agent.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.036769 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.036901 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: agent
May 07 15:25:25.036983 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: agent_resources_synced.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.037153 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.037310 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension agent-resources-synced not supported by any of loaded plugins
May 07 15:25:25.037379 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: agent_sort_key.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.037559 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Agents Sort Key" alias="agent_sort_key" description="Enabling the keyword sort_key for sorting in agents." updated="2013-02-03T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.037695 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension agent_sort_key not supported by any of loaded plugins
May 07 15:25:25.037756 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: allowedaddresspairs.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.038305 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.038449 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: allowed-address-pairs
May 07 15:25:25.038507 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: auto_allocated_topology.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.038716 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.038870 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: auto-allocated-topology
May 07 15:25:25.038937 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: availability_zone.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.039144 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.039293 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: availability_zone
May 07 15:25:25.039348 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: availability_zone_filter.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.039574 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.039705 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension availability_zone_filter not supported by any of loaded plugins
May 07 15:25:25.039782 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: data_plane_status.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.039959 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.040094 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension data-plane-status not supported by any of loaded plugins
May 07 15:25:25.040165 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: default_subnetpools.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.040359 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.040500 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: default-subnetpools
May 07 15:25:25.040571 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: dhcpagentscheduler.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.040860 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.040999 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: dhcp_agent_scheduler
May 07 15:25:25.041065 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: dns.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.041269 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.041406 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: dns-integration
May 07 15:25:25.041471 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: dns_domain_ports.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.041653 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.041784 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: dns-domain-ports
May 07 15:25:25.041857 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: dns_integration_domain_keywords.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.042042 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="DNS domain names with keywords allowed" alias="dns-integration-domain-keywords" description="Allows to use keywords like <project_id>, <project_name>, <user_id> and <user_name> as DNS domain name" updated="2020-06-15T18:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.042169 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: dns-integration-domain-keywords
May 07 15:25:25.042260 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: dvr.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.042474 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.042590 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension dvr not supported by any of loaded plugins
May 07 15:25:25.042650 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: ecmp_routes.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.042833 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Equal Cost Multipath Routing" alias="ecmp_routes" description="Allows traffic to reach thesame destination via multiple different links " updated="2020-11-26T18:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.042968 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension ecmp_routes not supported by any of loaded plugins
May 07 15:25:25.043039 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: empty_string_filtering.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.043238 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.043375 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: empty-string-filtering
May 07 15:25:25.043445 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.043676 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.043779 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins
May 07 15:25:25.043824 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.044001 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.044132 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins
May 07 15:25:25.044189 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: external_net.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.044391 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.044524 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: external-net
May 07 15:25:25.044594 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: extra_dhcp_opt.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.044773 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.044904 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: extra_dhcp_opt
May 07 15:25:25.044959 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: extraroute.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.045138 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.045299 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: extraroute
May 07 15:25:25.045367 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: extraroute_atomic.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.045551 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.045682 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: extraroute-atomic
May 07 15:25:25.045761 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: filter_validation.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.045938 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.046075 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: filter-validation
May 07 15:25:25.046146 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: fip_pf_description.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.046341 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.046478 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins
May 07 15:25:25.046547 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: fip_pf_detail.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.046723 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Floating IP port forwarding detail" alias="floating-ip-port-forwarding-detail" description="Allow list floating ip return more port forwarding data, include'id' and 'internal_port_id'" updated="2022-05-06T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.046854 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: floating-ip-port-forwarding-detail
May 07 15:25:25.046925 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: fip_pf_port_range.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.047098 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Floating IP Port Forwarding support port ranges" alias="floating-ip-port-forwarding-port-ranges" description="Add support to port ranges" updated="2020-07-01T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.047249 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension floating-ip-port-forwarding-port-ranges not supported by any of loaded plugins
May 07 15:25:25.047320 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: fip_port_details.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.047493 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.047636 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: fip-port-details
May 07 15:25:25.047706 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: flavors.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.048007 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.048143 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: flavors
May 07 15:25:25.048229 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.048556 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.048596 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins
May 07 15:25:25.048678 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: floatingip_pools.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.048893 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.049020 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: floatingip-pools
May 07 15:25:25.049091 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: ip_allocation.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.049295 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.049422 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension ip_allocation not supported by any of loaded plugins
May 07 15:25:25.049491 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: ip_substring_port_filtering.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.049666 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.049799 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension ip-substring-filtering not supported by any of loaded plugins
May 07 15:25:25.049869 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: l2_adjacency.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.050042 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.050172 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension l2_adjacency not supported by any of loaded plugins
May 07 15:25:25.050261 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: l3.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.050586 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.050723 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: router
May 07 15:25:25.050798 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: l3_conntrack_helper.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.051279 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.051415 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension l3-conntrack-helper not supported by any of loaded plugins
May 07 15:25:25.051488 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: l3_enable_default_route_bfd.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.051726 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Enable Default Route BFD" alias="enable-default-route-bfd" description="Enables configurable BFD behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.051863 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: enable-default-route-bfd
May 07 15:25:25.051930 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: l3_enable_default_route_ecmp.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.052115 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Enable Default Route ECMP" alias="enable-default-route-ecmp" description="Enables configurable ECMP behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.052261 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: enable-default-route-ecmp
May 07 15:25:25.052332 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: l3_ext_gw_mode.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.052508 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.052775 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: ext-gw-mode
May 07 15:25:25.052847 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: l3_ext_ha_mode.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.053017 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.053153 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: l3-ha
May 07 15:25:25.053238 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: l3_ext_ndp_proxy.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.053417 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Router support to enable NDP proxy" alias="l3-ext-ndp-proxy" description="Router extension to enable ndp proxy" updated="2020-08-25T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.053544 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension l3-ext-ndp-proxy not supported by any of loaded plugins
May 07 15:25:25.053613 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: l3_extra_gws.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.053792 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Neutron L3 External Gateway Multihoming" alias="external-gateway-multihoming" description="Allow multiple external gateway ports per router" updated="2023-01-18T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.053920 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: external-gateway-multihoming
May 07 15:25:25.053990 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: l3_flavors.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.054163 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.054314 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: l3-flavors
May 07 15:25:25.054384 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: l3_ndp_proxy.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.054615 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="name" alias="l3-ndp-proxy" description="description" updated="2021-06-29T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.054743 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension l3-ndp-proxy not supported by any of loaded plugins
May 07 15:25:25.054812 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.054988 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.055119 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension l3-port-ip-change-not-allowed not supported by any of loaded plugins
May 07 15:25:25.055187 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: l3agentscheduler.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.055739 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.055874 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension l3_agent_scheduler not supported by any of loaded plugins
May 07 15:25:25.055945 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: local_ip.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.056199 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Local IP" alias="local_ip" description="Support Node Local Virtual IP" updated="2021-07-26T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.056349 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension local_ip not supported by any of loaded plugins
May 07 15:25:25.056419 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: logging.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.056655 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.056782 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension logging not supported by any of loaded plugins
May 07 15:25:25.056853 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: metering.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.057225 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.057371 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension metering not supported by any of loaded plugins
May 07 15:25:25.057445 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.057634 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.057769 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension metering_source_and_destination_fields not supported by any of loaded plugins
May 07 15:25:25.057840 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: multiprovidernet.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.058019 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.058151 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: multi-provider
May 07 15:25:25.058243 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: netmtu.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.058420 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.058552 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: net-mtu
May 07 15:25:25.058621 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: netmtu_writable.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.058797 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.058929 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: net-mtu-writable
May 07 15:25:25.059002 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: network_availability_zone.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.059195 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.059344 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: network_availability_zone
May 07 15:25:25.059487 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: network_ha.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.059620 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Network HA creation flag" alias="network_ha" description="Network high availability creation flag." updated="2023-04-27T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.059750 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension network_ha not supported by any of loaded plugins
May 07 15:25:25.059820 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: network_ip_availability.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.060011 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.060143 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: network-ip-availability
May 07 15:25:25.060224 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: network_ip_availability_details.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.060439 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Network IP Availability Details Extension" alias="network-ip-availability-details" description="Add ip_availability_details attribute to network-ip-availabilities resource." updated="2025-10-31T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.060571 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: network-ip-availability-details
May 07 15:25:25.060641 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: network_segment_range.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.060883 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.061016 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension network-segment-range not supported by any of loaded plugins
May 07 15:25:25.061087 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: pagination.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.061287 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.061364 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: pagination
May 07 15:25:25.061433 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: port_device_profile.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.061604 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.061740 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: port-device-profile
May 07 15:25:25.061811 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: port_hardware_offload_type.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.061985 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port hardware offload" alias="port-hardware-offload-type" description="Expose port hardware offload extension" updated="2023-05-09T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.062119 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension port-hardware-offload-type not supported by any of loaded plugins
May 07 15:25:25.062189 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: port_hint_ovs_tx_steering.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.062385 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port hint: Open vSwitch Tx steering" alias="port-hint-ovs-tx-steering" description="Enable Port hint: Open vSwitch Tx steering" updated="2023-01-01T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.062521 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension port-hint-ovs-tx-steering not supported by any of loaded plugins
May 07 15:25:25.062590 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: port_hints.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.062762 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port hints" alias="port-hints" description="Backend specific port hints to allow backend specific tuning" updated="2023-01-01T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.062895 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension port-hints not supported by any of loaded plugins
May 07 15:25:25.062966 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: port_mac_address_override.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.063137 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Neutron Port MAC address override" alias="port-mac-override" description="Allow overriding the MAC address of a direct-physical Port via the active binding profile" updated="2022-03-18T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.063292 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension port-mac-override not supported by any of loaded plugins
May 07 15:25:25.063361 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: port_mac_address_regenerate.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.063554 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.063686 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: port-mac-address-regenerate
May 07 15:25:25.063756 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: port_numa_affinity_policy.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.063929 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.064063 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: port-numa-affinity-policy
May 07 15:25:25.064134 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: port_numa_affinity_policy_socket.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.064325 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port NUMA affinity policy "socket"" alias="port-numa-affinity-policy-socket" description="Adds "socket" to the supported port NUMA affinity policies" updated="2024-02-09T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.064456 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: port-numa-affinity-policy-socket
May 07 15:25:25.064676 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: port_resource_request.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.064828 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.064958 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension port-resource-request not supported by any of loaded plugins
May 07 15:25:25.065028 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: port_resource_request_groups.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.065477 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port Resource Request Groups" alias="port-resource-request-groups" description="Support requesting multiple groups of resources and traits from the same RP subtree in resource_request" updated="2021-08-02T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.065477 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension port-resource-request-groups not supported by any of loaded plugins
May 07 15:25:25.065477 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: port_trusted_vif.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.065861 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port trusted vif" alias="port-trusted-vif" description="Expose port 'trusted' attribute in the API" updated="2024-07-10T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.065861 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension port-trusted-vif not supported by any of loaded plugins
May 07 15:25:25.065861 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: portbindings.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.066001 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.066132 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: binding
May 07 15:25:25.066201 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: portbindings_extended.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.066420 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.066551 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: binding-extended
May 07 15:25:25.066621 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: portsecurity.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.066905 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.066957 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: port-security
May 07 15:25:25.066992 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: project_id.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.067165 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.067258 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: project-id
May 07 15:25:25.067328 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: providernet.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.067499 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.067657 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: provider
May 07 15:25:25.067728 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qinq.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.067919 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="QinQ networks" alias="qinq" description="Expose network 'qinq' attribute in the API. This allows to configure network to allow vlan in vlan configuration using 0x8a88 ethertype (QinQ)." updated="2024-12-02T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.068052 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: qinq
May 07 15:25:25.068120 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.068789 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.068929 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos not supported by any of loaded plugins
May 07 15:25:25.069001 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos_bw_limit_direction.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.069215 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.069361 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos-bw-limit-direction not supported by any of loaded plugins
May 07 15:25:25.069428 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.069659 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.069744 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins
May 07 15:25:25.069920 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos_default.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.069990 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.070125 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos-default not supported by any of loaded plugins
May 07 15:25:25.070708 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos_fip.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.070708 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.070708 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos-fip not supported by any of loaded plugins
May 07 15:25:25.070708 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos_fip_network_policy.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.070842 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="QoS floating IP network policy ID" alias="qos-fip-network-policy" description="Adds a the QoS network ID to the floating IP definition" updated="2021-11-15T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.070876 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos-fip-network-policy not supported by any of loaded plugins
May 07 15:25:25.070937 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos_gateway_ip.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.071106 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.071255 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos-gateway-ip not supported by any of loaded plugins
May 07 15:25:25.071328 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos_port_network_policy.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.071500 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.071660 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos-port-network-policy not supported by any of loaded plugins
May 07 15:25:25.071729 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos_pps_minimum_rule.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.071915 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="QoS minimum packet rate rule" alias="qos-pps-minimum" description="Allow to configure QoS minimum packet rate rule." updated="2021-07-14T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.072047 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos-pps-minimum not supported by any of loaded plugins
May 07 15:25:25.072116 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos_pps_minimum_rule_alias.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.072322 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="QoS minimum packet rate rule alias" alias="qos-pps-minimum-rule-alias" description="API to enable GET, PUT and DELETE operations on QoS minimum packet rate rule without specifying policy ID" updated="2021-10-22T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.072453 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos-pps-minimum-rule-alias not supported by any of loaded plugins
May 07 15:25:25.072523 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos_pps_rule.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.072713 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="QoS Rule Type Packet per Second Extension" alias="qos-pps" description="Add QoS Rule Type Packet per Second" updated="2021-05-12T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.072844 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos-pps not supported by any of loaded plugins
May 07 15:25:25.072914 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos_rule_type_details.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.073085 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.073234 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos-rule-type-details not supported by any of loaded plugins
May 07 15:25:25.073306 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos_rule_type_filter.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.073478 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Allow to filter the list of QoS rule types" alias="qos-rule-type-filter" description="Allows to filter the QoS rule type list adding two new flags. "all_rules" prints all implemented QoS rule types."all_supported" prints all supported QoS rule types by the loaded mechanism drivers" updated="2022-02-02T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.073611 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos-rule-type-filter not supported by any of loaded plugins
May 07 15:25:25.073681 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: qos_rules_alias.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.073862 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.073992 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension qos-rules-alias not supported by any of loaded plugins
May 07 15:25:25.074061 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: quota_check_limit.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.074254 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Quota engine limit check" alias="quota-check-limit" description="Support for checking the resource usage before applying a new quota limit" updated="2021-09-08T16:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.074386 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: quota-check-limit
May 07 15:25:25.074454 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: quota_check_limit_default.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.074661 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Quota engine limit check by default" alias="quota-check-limit-default" description="By default, the Neutron quota engine checks the resource usage before applying a new quota limit" updated="2024-08-21T16:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.074786 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: quota-check-limit-default
May 07 15:25:25.074855 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: quotasv2.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.075222 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per project" updated="2012-07-29T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.075436 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: quotas
May 07 15:25:25.075519 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: quotasv2_detail.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.075848 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.075925 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: quota_details
May 07 15:25:25.075996 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: rbac.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.076374 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control project access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.076518 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: rbac-policies
May 07 15:25:25.076586 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: rbac_address_group.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.076767 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.076901 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension rbac-address-group not supported by any of loaded plugins
May 07 15:25:25.076971 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: rbac_address_scope.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.077147 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.077301 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: rbac-address-scope
May 07 15:25:25.077371 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: rbac_security_groups.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.077547 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.077681 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: rbac-security-groups
May 07 15:25:25.077757 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: rbac_subnetpool.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.077931 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.078066 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension rbac-subnetpool not supported by any of loaded plugins
May 07 15:25:25.078137 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: revisionifmatch.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.078333 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.078464 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: revision-if-match
May 07 15:25:25.078535 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: revisions.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.078723 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.078865 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: standard-attr-revisions
May 07 15:25:25.078930 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: router_availability_zone.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.079124 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.079279 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: router_availability_zone
May 07 15:25:25.079350 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: router_enable_snat.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.079540 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Exposes the router "enable_snat" read-only configuration flag" alias="router-enable-snat" description="Exposes the router "enable_snat" read-only configuration flag" updated="2025-09-18T18:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.079674 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: router-enable-snat
May 07 15:25:25.079744 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: routerservicetype.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.079918 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.080048 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension router-service-type not supported by any of loaded plugins
May 07 15:25:25.080118 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: security_groups_default_rules.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.080461 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Default rules for security groups" alias="security-groups-default-rules" description="Configure set of security group rules used as default rules for every new security group" updated="2022-12-19T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.080597 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: security-groups-default-rules
May 07 15:25:25.080669 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.080850 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.080986 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: security-groups-normalized-cidr
May 07 15:25:25.081051 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: security_groups_port_filtering.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.081246 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.081375 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension port-security-groups-filtering not supported by any of loaded plugins
May 07 15:25:25.081445 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: security_groups_remote_address_group.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.081624 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.081765 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: security-groups-remote-address-group
May 07 15:25:25.081836 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: security_groups_rules_belongs_to_default_sg.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.082010 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Security group rule belongs to the project's default security group" alias="security-groups-rules-belongs-to-default-sg" description="Flag to determine if the security group rule belongs to the project's default security group" updated="2023-05-23T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.082143 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: security-groups-rules-belongs-to-default-sg
May 07 15:25:25.082214 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: security_groups_shared_filtering.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.082403 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Security group filtering on the shared field" alias="security-groups-shared-filtering" description="Support filtering security groups on the shared field" updated="2021-10-05T09:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.082535 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: security-groups-shared-filtering
May 07 15:25:25.082604 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: securitygroup.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.083895 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.084039 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: security-group
May 07 15:25:25.084111 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: segment.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.084369 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.084520 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension segment not supported by any of loaded plugins
May 07 15:25:25.084586 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.084771 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.084906 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins
May 07 15:25:25.084973 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: servicetype.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.085169 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.085318 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: service-type
May 07 15:25:25.085389 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: sorting.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.085576 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.085644 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: sorting
May 07 15:25:25.085829 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: standard_attr_segment.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.085948 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.086083 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension standard-attr-segment not supported by any of loaded plugins
May 07 15:25:25.086152 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: standardattrdescription.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.086369 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.086504 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: standard-attr-description
May 07 15:25:25.086573 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: stateful_security_group.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.086749 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.086883 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: stateful-security-group
May 07 15:25:25.086952 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: stdattrs_common.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.087108 np0000172501 neutron-ovn-maintenance-worker[104723]: WARNING neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py
May 07 15:25:25.087179 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.087376 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.087510 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: subnet-dns-publish-fixed-ip
May 07 15:25:25.087593 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: subnet_external_network.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.087765 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Subnet belongs to an external network" alias="subnet-external-network" description="Informs if the subnet belongs to an external network" updated="2024-02-05T18:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.087898 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: subnet-external-network
May 07 15:25:25.087967 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: subnet_onboard.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.088151 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.088404 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension subnet_onboard not supported by any of loaded plugins
May 07 15:25:25.088481 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: subnet_segmentid_writable.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.088662 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.088796 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension subnet-segmentid-writable not supported by any of loaded plugins
May 07 15:25:25.088867 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: subnet_service_types.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.089043 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.089179 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: subnet-service-types
May 07 15:25:25.089270 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: subnetallocation.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.089461 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.089596 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: subnet_allocation
May 07 15:25:25.089668 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.089863 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.090000 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: subnetpool-prefix-ops
May 07 15:25:25.090070 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: tag_creation.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.090267 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Tag creation extension" alias="tag-creation" description="Allow to create multiple tags for a resource" updated="2024-23-07T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.090402 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: tag-creation
May 07 15:25:25.090472 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.090649 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.090784 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins
May 07 15:25:25.090856 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: tagging.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.091434 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.091607 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: standard-attr-tag
May 07 15:25:25.091673 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: timestamp.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.091876 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.092012 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: standard-attr-timestamp
May 07 15:25:25.092086 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: trunk.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.092290 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.092433 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension trunk not supported by any of loaded plugins
May 07 15:25:25.092497 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: trunk_details.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.092672 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.092805 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension trunk-details not supported by any of loaded plugins
May 07 15:25:25.092875 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: uplink_status_propagation.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.093048 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.093181 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension uplink-status-propagation not supported by any of loaded plugins
May 07 15:25:25.093271 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: uplink_status_propagation_updatable.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.093447 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Uplink status propagation updatable extension" alias="uplink-status-propagation-updatable" description="Provides ability to update the port uplink status." updated="2024-09-03T18:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.093580 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Extension uplink-status-propagation-updatable not supported by any of loaded plugins
May 07 15:25:25.093651 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loading extension file: vlantransparent.py {{(pid=104723) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 07 15:25:25.093839 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=104723) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 07 15:25:25.094259 np0000172501 neutron-ovn-maintenance-worker[104723]: INFO neutron.api.extensions [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Loaded extension: vlan-transparent
May 07 15:25:25.096567 np0000172501 neutron-ovn-maintenance-worker[104723]: WARNING oslo_service.backend._threading.service [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Service MaintenanceWorker is not picklable with spawn; falling back to fork. Please make the service spawn-safe to avoid this fallback.
May 07 15:25:25.096567 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Full set of CONF: {{(pid=104723) _load_service_manager_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:59}}
May 07 15:25:25.096567 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ******************************************************************************** {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
May 07 15:25:25.096567 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Configuration options gathered from: {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
May 07 15:25:25.096946 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
May 07 15:25:25.096946 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
May 07 15:25:25.096946 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ================================================================================ {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
May 07 15:25:25.096946 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] agent_down_time                = 75 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.096946 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] allow_automatic_dhcp_failover  = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.097125 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] allow_bulk                     = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.097167 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.097286 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] api_extensions_path            =  {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.097377 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] api_paste_config               = api-paste.ini {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.097471 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] api_workers                    = 4 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.097556 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] auth_strategy                  = keystone {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.097645 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] base_mac                       = fa:16:3e:00:00:00 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.097746 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] bind_host                      = 0.0.0.0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.097836 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] bind_port                      = 9696 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.097923 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] client_socket_timeout          = 900 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.098015 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] config_dir                     = [] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.098083 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.098170 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] config_source                  = [] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.098285 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] control_exchange               = neutron {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.098369 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] core_plugin                    = ml2 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.098457 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] debug                          = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.098544 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] default_availability_zones     = [] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.098693 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.098771 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] dhcp_agent_notification        = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.098860 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] dhcp_agents_per_network        = 1 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.098943 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] dhcp_lease_duration            = 86400 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.099034 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] dhcp_load_type                 = networks {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.099121 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] dns_domain                     = openstacklocal {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.099220 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.099310 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] enable_default_route_bfd       = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.099395 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] enable_default_route_ecmp      = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.099480 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] enable_dvr                     = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.099598 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] enable_new_agents              = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.099678 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] enable_services_on_agents_with_admin_state_down = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.099762 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] enable_snat_by_default         = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.099845 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] enable_traditional_dhcp        = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.099927 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] external_dns_driver            = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.100010 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] filter_validation              = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.100099 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] global_physnet_mtu             = 1430 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.100163 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] graceful_shutdown_timeout      = 60 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.100297 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] host                           = np0000172501 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.100386 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] host_dvr_for_dhcp              = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.100471 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] http_retries                   = 3 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.100558 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] instance_format                = [instance: %(uuid)s]  {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.100646 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.100732 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ipam_driver                    = internal {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.100816 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ipv6_pd_enabled                = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.100899 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] l3_ha                          = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.100988 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.101077 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] l3_ha_network_physical_name    =  {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.101164 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] l3_ha_network_type             =  {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.101272 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] log_color                      = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.101361 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] log_config_append              = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.101451 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.101535 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] log_dir                        = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.101618 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] log_file                       = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.101682 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] log_options                    = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.101767 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] log_rotate_interval            = 1 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.101856 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] log_rotate_interval_type       = days {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.101943 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] log_rotation_type              = none {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.102009 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.102073 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.102161 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.102264 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.102327 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.102411 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] max_allowed_address_pair       = 10 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.102494 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] max_dns_nameservers            = 5 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.102576 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] max_header_line                = 16384 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.102658 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] max_l3_agents_per_router       = 3 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.102740 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] max_logfile_count              = 30 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.102822 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] max_logfile_size_mb            = 200 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.102905 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] max_routes                     = 30 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.102986 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] max_subnet_host_routes         = 20 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.103072 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] my_ip                          = 199.204.45.53 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.103158 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] my_ipv6                        = 2604:e100:1:0:f816:3eff:fe1e:3267 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.103262 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] network_auto_schedule          = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.103345 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] network_link_prefix            = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.103431 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.103517 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] notify_nova_on_port_data_changes = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.103618 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] notify_nova_on_port_status_changes = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.103706 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] pagination_max_limit           = -1 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.103790 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] periodic_fuzzy_delay           = 5 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.103872 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] periodic_interval              = 40 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.103955 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] publish_errors                 = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.104038 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] rate_limit_burst               = 0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.104125 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] rate_limit_except_level        = CRITICAL {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.104219 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] rate_limit_interval            = 0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.104309 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] router_distributed             = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.104394 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] rpc_resources_processing_step  = 20 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.104477 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] rpc_response_max_timeout       = 600 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.104561 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] rpc_response_timeout           = 60 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.104648 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] rpc_state_report_workers       = 0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.104730 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] rpc_workers                    = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.104821 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] send_events_interval           = 2 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.104912 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] service_plugins                = ['ovn-router'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.104999 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] setproctitle                   = on {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.105080 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] shell_completion               = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.105163 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] state_path                     = /opt/stack/data/neutron {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.105274 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] syslog_log_facility            = LOG_USER {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.105359 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] tcp_keepidle                   = 600 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.105446 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] transport_url                  = **** {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.105530 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] use_journal                    = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.105613 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] use_json                       = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.105695 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] use_stderr                     = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.105782 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] use_syslog                     = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.105864 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] vlan_qinq                      = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.105945 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] vlan_transparent               = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.106027 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] wsgi_default_pool_size         = 100 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.106110 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] wsgi_keep_alive                = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.106197 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.106301 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] wsgi_server_debug              = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.106401 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_concurrency.disable_process_locking = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.106499 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.106596 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler.connection_string     = messaging:// {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.106686 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler.enabled               = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.106776 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler.es_doc_type           = notification {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.106862 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler.es_scroll_size        = 10000 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.106952 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler.es_scroll_time        = 2m {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.107038 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler.filter_error_trace    = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.107126 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler.hmac_keys             = **** {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.107221 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler.sentinel_service_name = mymaster {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.107328 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler.socket_timeout        = 0.1 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.107416 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler.trace_requests        = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.107503 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler.trace_sqlalchemy      = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.107627 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler_jaeger.process_tags   = {} {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.107714 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler_jaeger.service_name_prefix = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.107801 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] profiler_otlp.service_name_prefix = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.107891 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.107977 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.108066 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.108156 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.108258 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.108356 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_policy.enforce_new_defaults = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.108523 np0000172501 neutron-ovn-maintenance-worker[104723]: WARNING oslo_config.cfg [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Deprecated: Option "enforce_scope" from group "oslo_policy" is deprecated for removal (This configuration was added temporarily to facilitate a smooth transition to the new RBAC. OpenStack will always enforce scope checks. This configuration option is deprecated and will be removed in the 2025.2 cycle.).  Its value may be silently ignored in the future.
May 07 15:25:25.108618 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_policy.enforce_scope      = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.108709 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_policy.policy_default_rule = default {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.108804 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.108894 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.108984 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.109067 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.109150 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.109250 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.109333 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.109421 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_policy.remote_timeout     = 60.0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.109515 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.109605 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] service_providers.service_provider = [] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.109697 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.auth_section              = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.109789 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.auth_type                 = password {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.109877 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.auth_url                  = https://199.204.45.53/identity {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.109962 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.cafile                    = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.110044 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.certfile                  = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.110131 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.collect_timing            = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.110229 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.default_domain_id         = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.110316 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.default_domain_name       = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.110399 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.domain_id                 = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.110485 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.domain_name               = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.110576 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.endpoint_type             = public {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.110663 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.insecure                  = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.110745 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.keyfile                   = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.110840 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.password                  = **** {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.110925 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.project_domain_id         = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.111011 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.project_domain_name       = Default {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.111099 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.project_id                = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.111189 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.project_name              = service {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.111300 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.region_name               = RegionOne {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.111387 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.split_loggers             = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.111469 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.system_scope              = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.111563 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.timeout                   = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.111650 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.trust_id                  = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.111734 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.user_domain_id            = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.111820 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.user_domain_name          = Default {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.111905 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.user_id                   = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.111995 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] nova.username                  = nova {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.112084 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] placement.auth_section         = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.112173 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] placement.auth_type            = password {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.112275 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] placement.cafile               = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.112359 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] placement.certfile             = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.112446 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] placement.collect_timing       = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.112535 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] placement.endpoint_type        = public {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.112620 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] placement.insecure             = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.112702 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] placement.keyfile              = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.112783 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] placement.region_name          = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.112868 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] placement.split_loggers        = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.112950 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] placement.timeout              = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.113037 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.auth_section            = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.113122 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.auth_type               = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.113215 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.cafile                  = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.113305 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.certfile                = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.113391 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.collect_timing          = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.113473 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.connect_retries         = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.113557 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.connect_retry_delay     = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.113642 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.enable_notifications    = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.113724 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.endpoint_override       = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.113809 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.insecure                = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.113891 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.interface               = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.113972 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.keyfile                 = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.114054 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.max_version             = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.114137 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.min_version             = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.114233 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.region_name             = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.114318 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.retriable_status_codes  = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.114401 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.service_name            = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.114482 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.service_type            = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.114564 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.split_loggers           = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.114646 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.status_code_retries     = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.114728 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.status_code_retry_delay = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.114815 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.timeout                 = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.114897 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.valid_interfaces        = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.114978 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ironic.version                 = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.115066 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] cli_script.dry_run             = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.115193 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.115298 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.115384 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.115469 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.115586 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.115672 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.115758 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.115843 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.115929 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.116018 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.hostname = np0000172501 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.116100 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.116188 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.116292 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.116379 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.116466 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.116558 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.116643 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.116728 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.116817 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.116901 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.116985 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.117070 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.117153 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.117256 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.117338 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.117506 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.117506 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.117632 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.117662 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.117740 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.117825 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.ssl      = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.117912 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.118001 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.118084 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.118171 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.118278 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.ssl_version =  {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.118384 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.118449 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_notifications.driver = [] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.118534 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_notifications.retry = -1 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.118626 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.118715 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_messaging_notifications.transport_url = **** {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.118798 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_reports.file_event_handler = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.118883 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_reports.file_event_handler_interval = 1 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.118964 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_reports.log_dir           = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.119053 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.119141 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] QUOTAS.default_quota           = -1 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.119249 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.119335 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] QUOTAS.quota_floatingip        = 50 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.119419 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] QUOTAS.quota_network           = 100 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.119503 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] QUOTAS.quota_port              = 500 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.119605 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] QUOTAS.quota_rbac_policy       = 10 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.119696 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] QUOTAS.quota_router            = 10 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.119779 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] QUOTAS.quota_security_group    = 10 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.119863 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] QUOTAS.quota_security_group_rule = 100 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.119945 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] QUOTAS.quota_subnet            = 100 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.120040 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] QUOTAS.track_quota_usage       = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.120132 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.120229 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep.group                  = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.120315 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep.helper_command         = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.120401 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep.log_daemon_traceback   = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.120489 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep.logger_name            = oslo_privsep.daemon {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.120575 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep.thread_pool_size       = 16 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.120658 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep.user                   = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.120747 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_dhcp_release.capabilities = [21, 12] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.120830 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_dhcp_release.group     = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.120911 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_dhcp_release.helper_command = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.120994 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_dhcp_release.log_daemon_traceback = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.121081 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.121164 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_dhcp_release.thread_pool_size = 16 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.121264 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_dhcp_release.user      = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.121363 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.121478 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_ovs_vsctl.group        = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.121513 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_ovs_vsctl.helper_command = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.121606 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.121703 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.121768 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.121850 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_ovs_vsctl.user         = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.121936 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_namespace.capabilities = [21] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.122018 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_namespace.group        = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.122099 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_namespace.helper_command = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.122183 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_namespace.log_daemon_traceback = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.122287 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.122371 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_namespace.thread_pool_size = 16 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.122453 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_namespace.user         = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.122544 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_conntrack.capabilities = [12] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.122625 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_conntrack.group        = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.122705 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_conntrack.helper_command = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.122789 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_conntrack.log_daemon_traceback = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.122874 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.122957 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_conntrack.thread_pool_size = 16 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.123038 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_conntrack.user         = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.123128 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_link.capabilities      = [12, 21] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.123217 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_link.group             = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.123307 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_link.helper_command    = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.123411 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_link.log_daemon_traceback = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.123474 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.123579 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_link.thread_pool_size  = 16 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.123665 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] privsep_link.user              = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.123758 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.123847 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.123932 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] AGENT.use_helper_for_ns_read   = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.124021 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] SECURITYGROUP.enable_ipset     = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.124110 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] SECURITYGROUP.enable_security_group = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.124192 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] SECURITYGROUP.firewall_driver  = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.124298 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] SECURITYGROUP.permitted_ethertypes = [] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.124387 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] experimental.ipv6_pd_enabled   = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.124490 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.124581 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ml2.external_network_type      = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.124673 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ml2.mechanism_drivers          = ['ovn'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.124763 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ml2.overlay_ip_version         = 4 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.124847 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ml2.path_mtu                   = 0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.124932 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ml2.physical_network_mtus      = {} {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.125026 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ml2.project_network_types      = ['geneve'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.125114 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.125250 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.125353 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ml2_type_flat.flat_networks    = ['public'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.125447 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ml2_type_geneve.max_header_size = 38 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.125540 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.125633 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.125724 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] network_log.burst_limit        = 25 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.125808 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] network_log.local_output_log_base = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.125892 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] network_log.rate_limit         = 100 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.125983 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.asyncio_connection    = **** {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.126066 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.asyncio_slave_connection = **** {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.126156 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.backend               = sqlalchemy {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.126261 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.connection            = **** {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.126347 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.connection_debug      = 0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.126435 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.connection_parameters =  {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.126521 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.connection_recycle_time = 3600 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.126606 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.connection_trace      = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.126690 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.db_inc_retry_interval = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.126774 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.db_max_retries        = 20 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.126857 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.db_max_retry_interval = 10 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.126941 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.db_retry_interval     = 1 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.127026 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.max_overflow          = 50 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.127109 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.max_pool_size         = 5 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.127192 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.max_retries           = 10 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.127301 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.127383 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.mysql_wsrep_sync_wait = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.127465 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.pool_timeout          = None {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.127549 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.retry_interval        = 10 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.127644 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.slave_connection      = **** {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.127728 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.sqlite_synchronous    = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.127813 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.synchronous_reader    = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.127896 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] database.use_db_reconnect      = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.127989 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.bfd_min_rx                 = 1000 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.128075 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.bfd_min_tx                 = 100 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.128157 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.bfd_mult                   = 3 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.128256 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.broadcast_arps_to_all_routers = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.128341 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.dhcp_default_lease_time    = 43200 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.128427 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.128512 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.dns_records_ovn_owned      = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.128599 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.dns_servers                = [] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.128683 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.enable_distributed_floating_ip = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.128767 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.fdb_age_threshold          = 300 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.128857 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ha_failover_strategy       = normal {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.128946 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.live_migration_activation_strategy = rarp {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.129056 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.localnet_learn_fdb         = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.129122 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.mac_binding_age_threshold  = 300 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.129212 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.neutron_sync_mode          = log {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.129303 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovn_dhcp4_global_options   = {} {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.129389 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovn_dhcp6_global_options   = {} {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.129480 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovn_l3_scheduler           = leastloaded {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.129569 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovn_metadata_enabled       = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.129657 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.129744 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.129836 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovn_nb_connection          = ['ssl:199.204.45.53:6641'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.129922 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.130007 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovn_router_indirect_snat   = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.130094 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.130230 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.130290 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovn_sb_connection          = ['ssl:199.204.45.53:6642'] {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.130377 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.130508 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovs_create_tap             = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.130536 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovsdb_connection_timeout   = 180 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.130628 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovsdb_log_level            = INFO {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.130711 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovsdb_probe_interval       = 60000 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.130795 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.ovsdb_retry_max_interval   = 180 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.130879 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.stateless_nat_enabled      = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.130966 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.131088 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] OVS.bridge_mac_table_size      = 50000 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.131132 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] OVS.igmp_flood                 = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.131231 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] OVS.igmp_flood_reports         = True {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.131315 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] OVS.igmp_flood_unregistered    = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.131404 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] OVS.igmp_snooping_enable       = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.131541 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] OVS.ovsdb_timeout              = 10 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.131598 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn_nb_global.fdb_removal_limit = 0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.131689 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn_nb_global.ignore_lsp_down  = False {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.131775 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.131839 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG cotyledon.oslo_config_glue [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] ******************************************************************************** {{(pid=104723) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
May 07 15:25:25.131920 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG oslo_service.backend._threading.service [None req-dd6b8272-631b-4616-bfc0-fe6facdb97ce None None] Selected the multiprocessing context: <multiprocessing.context.ForkContext object at 0x7bc77e3fed20> and updated it in Cotyledon ServiceManager: <multiprocessing.context.ForkContext object at 0x7bc77e3fed20> {{(pid=104723) launch_service /opt/stack/data/venv/lib/python3.12/site-packages/oslo_service/backend/_threading/service.py:394}}
May 07 15:25:25.167826 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] Full set of CONF: {{(pid=105503) _load_service_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:65}}
May 07 15:25:25.168380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
May 07 15:25:25.168380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] Configuration options gathered from: {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
May 07 15:25:25.168380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
May 07 15:25:25.168380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
May 07 15:25:25.168380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ================================================================================ {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
May 07 15:25:25.168773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] agent_down_time                = 75 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.168773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] allow_automatic_dhcp_failover  = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.168881 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] allow_bulk                     = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.169010 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.169075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] api_extensions_path            =  {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.169127 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] api_paste_config               = api-paste.ini {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.169265 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] api_workers                    = 4 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.169352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] auth_strategy                  = keystone {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.169430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] base_mac                       = fa:16:3e:00:00:00 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.169629 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] bind_host                      = 0.0.0.0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.169739 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] bind_port                      = 9696 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.169814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] client_socket_timeout          = 900 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.169927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] config_dir                     = [] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.169987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.170061 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] config_source                  = [] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.170152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] control_exchange               = neutron {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.170244 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] core_plugin                    = ml2 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.170346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] debug                          = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.170424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] default_availability_zones     = [] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.170604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.170755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agent_notification        = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.170755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agents_per_network        = 1 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.170860 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] dhcp_lease_duration            = 86400 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.170926 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] dhcp_load_type                 = networks {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.170981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] dns_domain                     = openstacklocal {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.171080 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.171146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_bfd       = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.171235 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_ecmp      = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.171296 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] enable_dvr                     = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.171351 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] enable_new_agents              = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.171425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] enable_services_on_agents_with_admin_state_down = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.171504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] enable_snat_by_default         = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.171596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] enable_traditional_dhcp        = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.171665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] external_dns_driver            = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.171709 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] fatal_deprecations             = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.171768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] filter_validation              = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.171845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] global_physnet_mtu             = 1430 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.171912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] graceful_shutdown_timeout      = 60 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.172165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] host                           = np0000172501 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.172312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] host_dvr_for_dhcp              = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.172381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] http_retries                   = 3 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.172450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] instance_format                = [instance: %(uuid)s]  {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.172517 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.172592 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ipam_driver                    = internal {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.172658 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ipv6_pd_enabled                = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.172746 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] l3_ha                          = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.172825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.172900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_physical_name    =  {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.172966 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_type             =  {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] log_color                      = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173129 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] log_config_append              = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173289 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] log_dir                        = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173353 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] log_file                       = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] log_options                    = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173464 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval            = 1 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173543 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval_type       = days {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] log_rotation_type              = none {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173730 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173780 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.173991 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] max_allowed_address_pair       = 10 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.174061 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] max_dns_nameservers            = 5 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.174123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] max_header_line                = 16384 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.174346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] max_l3_agents_per_router       = 3 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.174409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_count              = 30 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.174534 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_size_mb            = 200 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.174674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] max_routes                     = 30 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.174789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] max_subnet_host_routes         = 20 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.174858 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] my_ip                          = 199.204.45.53 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.174945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] my_ipv6                        = 2604:e100:1:0:f816:3eff:fe1e:3267 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.175031 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] network_auto_schedule          = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.175098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] network_link_prefix            = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.175183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.175278 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_data_changes = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.175377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_status_changes = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.175460 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] pagination_max_limit           = -1 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.175535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] periodic_fuzzy_delay           = 5 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.175628 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] periodic_interval              = 40 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.175699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] publish_errors                 = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.175762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_burst               = 0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.175830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_except_level        = CRITICAL {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.175893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_interval            = 0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.175975 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] router_distributed             = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.176048 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] rpc_resources_processing_step  = 20 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.176126 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_max_timeout       = 600 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.176201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_timeout           = 60 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.176287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] rpc_state_report_workers       = 0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.176350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] rpc_workers                    = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.176413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] send_events_interval           = 2 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.176499 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] service_plugins                = ['ovn-router'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.176575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] setproctitle                   = on {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.176645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] shell_completion               = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.176723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] state_path                     = /opt/stack/data/neutron {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.176801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] syslog_log_facility            = LOG_USER {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.176865 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] tcp_keepidle                   = 600 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.176939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] transport_url                  = **** {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.177005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] use_journal                    = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.177069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] use_json                       = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.177130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] use_stderr                     = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.177197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] use_syslog                     = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.177276 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] vlan_qinq                      = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.177338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] vlan_transparent               = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.177408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] wsgi_default_pool_size         = 100 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.177471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] wsgi_keep_alive                = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.177562 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.177637 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] wsgi_server_debug              = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 07 15:25:25.177760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.disable_process_locking = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.177862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.177987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler.connection_string     = messaging:// {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.178081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler.enabled               = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.178161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_doc_type           = notification {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.178258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_size        = 10000 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.178335 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_time        = 2m {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.178681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler.filter_error_trace    = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.178681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler.hmac_keys             = **** {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.178681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler.sentinel_service_name = mymaster {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.178681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler.socket_timeout        = 0.1 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.178809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_requests        = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.178809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_sqlalchemy      = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.178870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.process_tags   = {} {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.178946 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.service_name_prefix = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179012 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] profiler_otlp.service_name_prefix = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_process_name =  {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_new_defaults = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179648 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_scope      = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179706 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_default_rule = default {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179940 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.179977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180013 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180056 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_timeout     = 60.0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180135 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] service_providers.service_provider = [] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_section              = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_type                 = password {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_url                  = https://199.204.45.53/identity {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.cafile                    = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.certfile                  = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.collect_timing            = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180587 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_id         = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180633 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_name       = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_id                 = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180707 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_name               = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.endpoint_type             = public {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.insecure                  = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.keyfile                   = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180890 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.password                  = **** {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180930 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_id         = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.180971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_name       = Default {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.project_id                = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.project_name              = service {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.region_name               = RegionOne {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181138 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.split_loggers             = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181171 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.system_scope              = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181243 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.timeout                   = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181286 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.trust_id                  = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_id            = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_name          = Default {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.user_id                   = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181451 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] nova.username                  = nova {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_section         = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181557 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_type            = password {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] placement.cafile               = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] placement.certfile             = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181676 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] placement.collect_timing       = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] placement.endpoint_type        = public {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] placement.insecure             = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] placement.keyfile              = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181852 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] placement.region_name          = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] placement.split_loggers        = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181937 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] placement.timeout              = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.181987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_section            = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_type               = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182078 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.cafile                  = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.certfile                = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182154 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.collect_timing          = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182212 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retries         = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182295 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retry_delay     = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.enable_notifications    = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.endpoint_override       = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.insecure                = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182466 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.interface               = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.keyfile                 = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182548 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.max_version             = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.min_version             = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.region_name             = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.retriable_status_codes  = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_name            = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_type            = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.split_loggers           = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retries     = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retry_delay = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182934 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.timeout                 = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.182987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.valid_interfaces        = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ironic.version                 = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183080 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] cli_script.dry_run             = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183219 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183278 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183433 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183527 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183558 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183628 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.hostname = np0000172501 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183794 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183934 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.183978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184061 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184180 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184246 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184330 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184445 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl      = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184587 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184726 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_key_file =  {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184759 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_version =  {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184794 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.use_queue_manager = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.driver = [] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.184951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.retry = -1 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185008 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185054 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.transport_url = **** {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185217 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.log_dir           = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185289 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.default_quota           = -1 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_floatingip        = 50 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185517 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_network           = 100 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185568 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_port              = 500 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185609 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_rbac_policy       = 10 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_router            = 10 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group    = 10 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group_rule = 100 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_subnet            = 100 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.185957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.track_quota_usage       = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186052 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep.group                  = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep.helper_command         = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186235 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep.log_daemon_traceback   = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186315 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep.logger_name            = oslo_privsep.daemon {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep.thread_pool_size       = 16 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep.user                   = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.group     = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.helper_command = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.log_daemon_traceback = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186670 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.thread_pool_size = 16 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186719 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.user      = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186803 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.group        = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.helper_command = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.186957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187009 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.user         = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.capabilities = [21] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.group        = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187151 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.helper_command = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.log_daemon_traceback = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187250 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.thread_pool_size = 16 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.user         = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.capabilities = [12] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.group        = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.helper_command = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187511 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.log_daemon_traceback = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187618 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.thread_pool_size = 16 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.user         = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187727 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.capabilities      = [12, 21] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.group             = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187817 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.helper_command    = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.log_daemon_traceback = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187906 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.thread_pool_size  = 16 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.187981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.user              = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] AGENT.use_helper_for_ns_read   = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_ipset     = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188278 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_security_group = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.firewall_driver  = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] experimental.ipv6_pd_enabled   = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188516 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188562 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ml2.external_network_type      = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188600 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ml2.mechanism_drivers          = ['ovn'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ml2.overlay_ip_version         = 4 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188671 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ml2.path_mtu                   = 0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188725 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ml2.physical_network_mtus      = {} {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188765 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ml2.project_network_types      = ['geneve'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_flat.flat_networks    = ['public'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.188988 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.max_header_size = 38 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189022 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189153 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] network_log.burst_limit        = 25 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189216 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] network_log.local_output_log_base = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189269 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] network_log.rate_limit         = 100 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_connection    = **** {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_slave_connection = **** {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.backend               = sqlalchemy {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189494 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.connection            = **** {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.connection_debug      = 0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189586 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.connection_parameters =  {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.connection_recycle_time = 3600 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.connection_trace      = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189727 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.db_inc_retry_interval = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retries        = 20 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retry_interval = 10 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.db_retry_interval     = 1 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.max_overflow          = 50 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.max_pool_size         = 5 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.189992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.max_retries           = 10 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_sql_mode        = TRADITIONAL {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190084 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_wsrep_sync_wait = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.pool_timeout          = None {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190160 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.retry_interval        = 10 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190242 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.slave_connection      = **** {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190295 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.sqlite_synchronous    = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.synchronous_reader    = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] database.use_db_reconnect      = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190451 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_rx                 = 1000 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_tx                 = 100 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_mult                   = 3 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190574 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.broadcast_arps_to_all_routers = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190621 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.dhcp_default_lease_time    = 43200 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190655 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190704 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_records_ovn_owned      = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_servers                = [] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.enable_distributed_floating_ip = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.fdb_age_threshold          = 300 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ha_failover_strategy       = normal {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.live_migration_activation_strategy = rarp {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.190964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.localnet_learn_fdb         = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.mac_binding_age_threshold  = 300 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.neutron_sync_mode          = log {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp4_global_options   = {} {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp6_global_options   = {} {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_l3_scheduler           = leastloaded {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191250 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_metadata_enabled       = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_connection          = ['ssl:199.204.45.53:6641'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191451 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191488 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_router_indirect_snat   = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191631 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_connection          = ['ssl:199.204.45.53:6642'] {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191707 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovs_create_tap             = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_connection_timeout   = 180 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_log_level            = INFO {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191835 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_probe_interval       = 60000 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_retry_max_interval   = 180 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.stateless_nat_enabled      = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.191974 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.192030 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] OVS.bridge_mac_table_size      = 50000 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.192081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood                 = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.192131 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_reports         = True {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.192181 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_unregistered    = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.192249 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_snooping_enable       = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.192303 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] OVS.ovsdb_timeout              = 10 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.192394 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.fdb_removal_limit = 0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.192394 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.ignore_lsp_down  = False {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 07 15:25:25.192493 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=105503) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
May 07 15:25:25.192698 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG cotyledon._service_worker [-] Run service ServiceWrapper(0) [105503] {{(pid=105503) wait_forever /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/_service_worker.py:218}}
May 07 15:25:25.194575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-232761', 'neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._post_fork_initialize-115133'] for process (None), after_init {{(pid=105503) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 07 15:25:25.195866 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbNbOvnIdl for MaintenanceWorker with retry
May 07 15:25:25.247547 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Switch.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 07 15:25:25.247547 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Router.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 07 15:25:25.247547 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Load_Balancer.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 07 15:25:25.247902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Copp.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.247902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Meter.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.247902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Load_Balancer_Group.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.247902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index HA_Chassis_Group.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.247902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Address_Set.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.247902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Connection.target {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.247902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Gateway_Chassis.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.248504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Mirror.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.248504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Template_Var.chassis {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.248504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Router_Port.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.248504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Port_Group.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.248504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Switch_Port.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.251410 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.53:6641: connecting...
May 07 15:25:25.257131 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.53:6641: connected
May 07 15:25:25.282768 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbSbOvnIdl for MaintenanceWorker with retry
May 07 15:25:25.308337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Chassis.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 07 15:25:25.308479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Port_Binding.logical_port {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 07 15:25:25.308479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Private.name {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.308479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Datapath_Binding.tunnel_key {{(pid=105503) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 07 15:25:25.310703 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.53:6642: connecting...
May 07 15:25:25.315822 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.53:6642: connected
May 07 15:25:25.351479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7bc778d40530>> floatingip precommit_update 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.351617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7bc778d40530>> floatingip precommit_delete 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.351724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7bc778d40530>> floatingip before_create 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.351724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7bc778d40530>> floatingip before_update 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.351944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7bc778d40530>> port after_update 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.351944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7bc778d40530>> port precommit_delete 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.352099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding.register-115025'] for port_forwarding_plugin (None), after_init {{(pid=105503) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 07 15:25:25.352183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7bc778ceb650>> port_forwarding after_create 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.352295 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7bc778ceb650>> router_interface after_create 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.352374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7bc778ceb650>> port_forwarding after_update 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.352374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7bc778ceb650>> port_forwarding after_delete 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.352558 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7bc778ceb650>> router_interface after_delete 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.352558 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 07 15:25:25.352726 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.placement [-] Starting OVNClientPlacementExtension
May 07 15:25:25.353038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._prevent_segment_delete_with_subnet_associated of <neutron.services.segments.plugin.Plugin object at 0x7bc7793e24e0>> segment before_delete 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.353142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._validate_auto_address_subnet_delete of <neutron.services.segments.plugin.Plugin object at 0x7bc7793e24e0>> subnet precommit_delete_associations 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.353414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_addition_to_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7bc7793e25a0>> segment_host_mapping after_create 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.353508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_removal_from_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7bc7793e25a0>> segment_host_mapping after_delete 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.353508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7bc7793e25a0>> port after_create 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.353694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7bc7793e25a0>> port after_delete 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.353694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7bc7793e25a0>> port after_update 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.353694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_segment_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7bc7793e25a0>> segment after_delete 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.353930 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_created of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7bc7793e25a0>> subnet after_create 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.353930 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7bc7793e25a0>> subnet after_delete 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.353930 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7bc7793e25a0>> subnet after_update 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.404251 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7bc779423e00>> subnet after_create 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.404484 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_delete of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7bc779423e00>> subnet after_delete 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.404484 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7bc779423e00>> subnet before_create 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.404484 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_update of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7bc779423e00>> subnet before_update 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.406740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_resource_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x7bc7793cb920>> security_group after_delete 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.406833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_target_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x7bc7793cb920>> port after_delete 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.406910 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.logapi.drivers.ovn.driver.DriverBase._register-1431795'] for logging-plugin (None), after_init {{(pid=105503) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 07 15:25:25.407013 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7bc779423e90>> security_group_rule after_create 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.407102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7bc779423e90>> security_group_rule after_update 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:25.407186 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7bc779423e90>> security_group_rule after_delete 55550000, False {{(pid=105503) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 07 15:25:34.902011 np0000172501 neutron-ovn-maintenance-worker[104723]: DEBUG dbcounter [-] [104723] Writing DB stats neutron:SELECT=4 {{(pid=104723) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:25:35.408365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Northbound DB sync process started @ 2026-05-07 15:25:35.407922 {{(pid=105503) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:99}}
May 07 15:25:35.411019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync port groups started @ 2026-05-07 15:25:35.410539 {{(pid=105503) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:132}}
May 07 15:25:35.427032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writer thread running {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
May 07 15:25:35.438755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync port groups completed @ 2026-05-07 15:25:35.438442 {{(pid=105503) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:196}}
May 07 15:25:35.438885 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync networks, ports and DHCP options started @ 2026-05-07 15:25:35.438721 {{(pid=105503) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1152}}
May 07 15:25:35.747508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.services.segments.db [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] neutron.services.segments.plugin.Plugin method get_segments called with arguments (<neutron_lib.context.Context object at 0x7bc7791cf500>,) {'filters': {'network_id': ['8b456c85-5e2d-47f8-9886-ebf65932cf32'], 'is_dynamic': False}} {{(pid=105503) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
May 07 15:25:35.751273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync metadata ports started {{(pid=105503) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1097}}
May 07 15:25:35.810412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync metadata ports completed {{(pid=105503) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1149}}
May 07 15:25:35.810412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync DHCP options for Neutron subnets started {{(pid=105503) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:949}}
May 07 15:25:35.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync DHCP options for Neutron subnets completed @ 2026-05-07 15:25:35.863530 {{(pid=105503) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1022}}
May 07 15:25:35.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 07 15:25:35.864959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned started {{(pid=105503) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1027}}
May 07 15:25:35.865070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned completed {{(pid=105503) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1086}}
May 07 15:25:35.865268 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync networks, ports and DHCP options completed @ 2026-05-07 15:25:35.865062 {{(pid=105503) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1321}}
May 07 15:25:35.865420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync ACLs started @ 2026-05-07 15:25:35.865275 {{(pid=105503) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:239}}
May 07 15:25:35.899866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync ACLs completed @ 2026-05-07 15:25:35.899530 {{(pid=105503) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:431}}
May 07 15:25:35.899866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync routers and router ports started @ 2026-05-07 15:25:35.899740 {{(pid=105503) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:736}}
May 07 15:25:35.949467 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 07 15:25:35.949813 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync routers and router ports completed @ 2026-05-07 15:25:35.949561 {{(pid=105503) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:944}}
May 07 15:25:35.949934 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync port QoS policies started @ 2026-05-07 15:25:35.949807 {{(pid=105503) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1369}}
May 07 15:25:35.950059 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] Starting OVNClientQosExtension
May 07 15:25:35.965172 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync port QoS policies completed @ 2026-05-07 15:25:35.965012 {{(pid=105503) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1392}}
May 07 15:25:35.965319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync Floating IP QoS policies started @ 2026-05-07 15:25:35.965143 {{(pid=105503) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1397}}
May 07 15:25:35.965403 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] Starting OVNClientQosExtension
May 07 15:25:35.985311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync Floating IP QoS policies completed @ 2026-05-07 15:25:35.985121 {{(pid=105503) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1420}}
May 07 15:25:35.985444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync Floating IP NAT rules started @ 2026-05-07 15:25:35.985277 {{(pid=105503) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1425}}
May 07 15:25:35.985568 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-NB Sync Floating IP NAT rules completed @ 2026-05-07 15:25:35.985470 {{(pid=105503) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1451}}
May 07 15:25:35.985654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-0435f37a-6b0f-4d85-9220-d0b77a871630 None None] OVN-Northbound DB sync process completed @ 2026-05-07 15:25:35.985564 {{(pid=105503) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:112}}
May 07 15:25:45.409520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Southbound DB sync process started @ 2026-05-07 15:25:45.409112 {{(pid=105503) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1480}}
May 07 15:25:45.410979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-89181129-6dbc-4bd5-aead-851a17cd3e8f None None] OVN-SB Sync hostname and physical networks started @ 2026-05-07 15:25:45.410696 {{(pid=105503) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1511}}
May 07 15:25:45.412478 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 07 15:25:45.419395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.419395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.419395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_dhcp_options_consistency {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.419395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fair_meter_consistency {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.419395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fdb_aging_settings {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.419909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_igmp_snoop_support {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.419909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_inconsistencies {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.419909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_global_dhcp_opts {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.419909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.419909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.419909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_provider_distributed_ports {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.420346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.420346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.420346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.420346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.configure_nb_global {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.420346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.420346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_fip_distributed_flag {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.420963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_network_type_and_physnet {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.420963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_ovn_owned_dns_option {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.420963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_ha_failover {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.420963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.420963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_mac_aging_settings {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.420963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_distributed_flag {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.421408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_static_routes {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.421408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_security_group_with_address_group {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.421408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_virtual_port_parent_hostname {{(pid=105503) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 07 15:25:45.422765 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-89181129-6dbc-4bd5-aead-851a17cd3e8f None None] Agents populated in the agent cache: {{(pid=105503) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:310}}
May 07 15:25:45.422934 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-89181129-6dbc-4bd5-aead-851a17cd3e8f None None] - Agent type: OVN Controller Gateway agent, hostname: np0000172501, name: 6c608f94-302c-45c4-8e1f-fa23d1659434 {{(pid=105503) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:312}}
May 07 15:25:45.423098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-89181129-6dbc-4bd5-aead-851a17cd3e8f None None] - Agent type: OVN Neutron agent, hostname: np0000172501, name: 6c608f94-302c-45c4-8e1f-fa23d1659434 {{(pid=105503) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:312}}
May 07 15:25:45.424333 np0000172501 neutron-ovn-maintenance-worker[105503]: WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-89181129-6dbc-4bd5-aead-851a17cd3e8f None None] New host np0000172501 found in OVN SB DB, but not in Neutron. Add its SegmentHostMapping in Neutron
May 07 15:25:45.424932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.425913 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] Maintenance task thread has started
May 07 15:25:45.426039 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] MaintenanceWorker process has finished the post initialization
May 07 15:25:45.428155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.428872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.429107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.429650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.432572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.432973 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.433112 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.433187 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.433309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.433401 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.433475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.433581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.433692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.433773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.433863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.433957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.434037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.434130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.434287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.434366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.434462 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.434543 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.434606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.434707 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.434782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' {{(pid=105503) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 07 15:25:45.443138 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG oslo_concurrency.lockutils [None req-89181129-6dbc-4bd5-aead-851a17cd3e8f None None] Acquiring lock "update_segment_host_mapping" by "neutron.services.segments.db.update_segment_host_mapping" {{(pid=105503) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
May 07 15:25:45.443387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG oslo_concurrency.lockutils [None req-89181129-6dbc-4bd5-aead-851a17cd3e8f None None] Lock "update_segment_host_mapping" acquired by "neutron.services.segments.db.update_segment_host_mapping" :: waited 0.000s {{(pid=105503) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
May 07 15:25:45.462889 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron_lib.callbacks.manager [None req-89181129-6dbc-4bd5-aead-851a17cd3e8f None None] Publish callbacks ['neutron.plugins.ml2.plugin.DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments-23381', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_host_addition_to_aggregate-608548'] for segment_host_mapping (None), after_create {{(pid=105503) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 07 15:25:45.468664 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:45.470409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_baremetal_ports_dhcp_options {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:45.470522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Sync baremetal port DHCP options with configuration. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:45.493488 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.services.segments.db [None req-89181129-6dbc-4bd5-aead-851a17cd3e8f None None] Segments {'2b8b3cbc-130c-4894-94e1-ab0729ec7077'} mapped to the host np0000172501 {{(pid=105503) update_segment_host_mapping /opt/stack/neutron/neutron/services/segments/db.py:249}}
May 07 15:25:45.510279 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG oslo_concurrency.lockutils [None req-89181129-6dbc-4bd5-aead-851a17cd3e8f None None] Lock "update_segment_host_mapping" "released" by "neutron.services.segments.db.update_segment_host_mapping" :: held 0.067s {{(pid=105503) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
May 07 15:25:45.510422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-89181129-6dbc-4bd5-aead-851a17cd3e8f None None] OVN-SB Sync hostname and physical networks completed @ 2026-05-07 15:25:45.510159 {{(pid=105503) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1538}}
May 07 15:25:45.526738 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_baremetal_ports_dhcp_options finished in 0.056 seconds
May 07 15:25:45.526941 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:45.527070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_dhcp_options_consistency {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:45.527144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Backfill network_id on DHCP_Options external_ids. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:45.571254 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_dhcp_options_consistency finished in 0.044 seconds
May 07 15:25:45.571443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:45.571642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fair_meter_consistency {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:45.571736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Refresh ACL logging fair meter after configuration reload. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:45.572067 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fair_meter_consistency finished in 0.000 seconds
May 07 15:25:45.572254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:45.572608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:45.572705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:45.590992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:25:45.705821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:45.705821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:45.707075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:45.707075 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:45.707075 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:45.707075 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:45.707075 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:45.707075 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:45.707075 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:45.707075 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:45.720420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.134 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:45.720420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:45.720420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:45.720420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:45.720420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:45.720420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:25:45.720420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:45.720420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:45.720420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:45.720420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:45.720420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:45.720420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:45.720996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:45.720996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:45.720996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:45.720996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:45.720996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:45.720996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:45.720996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:45.720996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:45.720996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:45.720996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:45.720996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:45.720996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:45.720996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:45.721553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:45.721553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:45.724015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:45.725412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:45.726911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:45.726911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:45.726911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:45.727384 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_for_igmp_snoop_support {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:45.727511 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Check for IGMP snoop support. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:45.728127 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_for_igmp_snoop_support finished in 0.000 seconds
May 07 15:25:45.728316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:45.728471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:45.728586 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:45.757269 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: No inconsistencies found. Skipping {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:437}}
May 07 15:25:45.757482 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_for_inconsistencies finished in 0.029 seconds
May 07 15:25:45.757740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_global_dhcp_opts {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:45.757864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Check global DHCP options consistency. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:45.758085 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_global_dhcp_opts finished in 0.000 seconds
May 07 15:25:45.758232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:45.758498 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_localnet_port_has_learn_fdb {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:45.758614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Ensure localnet ports have learn-fdb set per configuration. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:45.762256 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 07 15:25:45.764534 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_localnet_port_has_learn_fdb finished in 0.006 seconds
May 07 15:25:45.764656 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:45.764837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_network_broadcast_arps_to_all_routers {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:45.764927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Sync broadcast-arps-to-all-routers on external networks. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:45.823480 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_network_broadcast_arps_to_all_routers finished in 0.058 seconds
May 07 15:25:45.823742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:45.823935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_provider_distributed_ports {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:45.824039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Align reside-on-redirect-chassis on VLAN/FLAT distributed ports. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:45.939843 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_provider_distributed_ports finished in 0.115 seconds
May 07 15:25:45.940148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:45.940603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_redirect_type_router_gateway_ports {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:45.942190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Align router gateway redirect-type for provider VLAN/FLAT networks. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:45.978264 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 07 15:25:45.978646 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-89181129-6dbc-4bd5-aead-851a17cd3e8f None None] OVN-Southbound DB sync process completed @ 2026-05-07 15:25:45.978339 {{(pid=105503) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1488}}
May 07 15:25:46.031545 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_redirect_type_router_gateway_ports finished in 0.089 seconds
May 07 15:25:46.031692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:46.031894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:46.032020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:46.034327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:46.034327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.035070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:46.035070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.035070 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:46.035070 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:46.035070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.035070 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:46.035070 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.035070 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:46.036137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.036137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:46.036137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:46.036137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:46.036137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.036137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:25:46.036137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:46.036137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.036137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:46.036137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.036137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.036137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.037080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:46.037080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.037080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:46.037080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.037080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:46.037080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:46.037080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.037080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:46.037080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.037080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:46.037080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:46.037080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.037080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:46.037926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:46.038962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.040176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:46.040176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:46.040176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.040176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:46.040176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.040176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:46.040176 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: cleanup_old_hash_ring_nodes {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:46.040176 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Purge stale hash ring nodes older than five days. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:46.042072 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.db.ovn_hash_ring_db [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Cleaned up Hash Ring nodes older than 5 days
May 07 15:25:46.042200 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task cleanup_old_hash_ring_nodes finished in 0.005 seconds
May 07 15:25:46.042427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: configure_nb_global {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:46.042581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply ovn_nb_global settings to NB_Global options. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:46.042862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:25:46.043494 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:46.043987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.044823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:46.044823 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.044823 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:46.044823 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:46.044823 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.044823 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:46.044823 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.044823 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:25:46.046158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:25:46.046899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.046899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:25:46.046899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.046899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:25:46.046899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:25:46.046899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.046899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:25:46.046899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.046899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:25:46.047604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task configure_nb_global failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.047604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:46.047604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:46.047604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:46.047604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.047604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global
May 07 15:25:46.047604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:25:46.047604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:46.047604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:46.047604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.047604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:46.048072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' (it runs every 86400.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:46.048775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:46.049383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.050142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:46.050142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:46.050142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.050142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:46.050142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.050142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:46.050142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:46.050142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.050142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:46.050142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.050142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:46.050142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 07 15:25:46.050142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:46.051004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:46.051004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:46.051004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:25:46.051159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:46.051494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.052345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:46.052345 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.052345 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:46.052345 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:46.052345 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.052345 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:46.052345 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.052345 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:25:46.053503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:25:46.054246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.054246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:25:46.054246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.054246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:25:46.054246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:25:46.054246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.054246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:25:46.054246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.054246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:25:46.054731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.054731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:46.054731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:46.054731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:46.054731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.054731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:25:46.054731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:25:46.054731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:25:46.054731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:46.054731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:46.054731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.054731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:46.055282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:46.055802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.055802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:46.055802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.055802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:46.056327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.057166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:46.057911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.057911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:46.057911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:46.057911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.057911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:46.057911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.057911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:46.057911 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_network_type_and_physnet {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:46.057911 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set network type and physnet external_ids on logical switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:46.063943 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_network_type_and_physnet finished in 0.007 seconds
May 07 15:25:46.064129 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:46.064331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_ovn_owned_dns_option {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:46.064455 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply ovn_owned option on DNS records per configuration. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:46.064724 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_ovn_owned_dns_option finished in 0.000 seconds
May 07 15:25:46.064855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:46.064995 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:46.065111 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:46.065371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:25:46.065965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:46.066393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.067184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:46.067184 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.067184 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:46.067184 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:46.067184 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.067184 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:46.067184 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.067184 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:25:46.068201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:25:46.068884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.068884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:25:46.068884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.068884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:25:46.068884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:25:46.068884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.068884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:25:46.068884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.068884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:25:46.069246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.069246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:46.069246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:46.069246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:46.069246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.069246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:25:46.069246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:25:46.069246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:46.069246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:46.069246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.069246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:46.069694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:46.070191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.070191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:46.070191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.070191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.070369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:46.071071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:46.071788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.071788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:46.071788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.071788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:46.071788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_lrouter_ports_ext_ids_name_prefix {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:46.071788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: ('Check missing prefix router_name', 'in external_ids of LRPs.') {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:46.072220 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_lrouter_ports_ext_ids_name_prefix finished in 0.001 seconds
May 07 15:25:46.072452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:46.072676 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:46.072836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:46.073074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:25:46.073705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:46.074134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:46.074410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.075170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:46.075170 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.075170 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:46.075170 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:46.075170 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.075170 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:46.075170 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.075170 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:46.076169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.076169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:46.076169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:46.076169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:46.076169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.076169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:25:46.076169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:46.076169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.076169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:46.076169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.076169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.076169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.076673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:46.076673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.076673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:46.076673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.076673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:46.076673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:46.076673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.076673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:46.076673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.076673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:46.076673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:46.076673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.076673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:46.077133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:46.077690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.078322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:46.078322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:46.078322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.078322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:46.078322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.078322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:46.078322 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_router_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:46.078322 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Sync router.distributed with distributed floating IP config. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:46.081269 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_router_distributed_flag finished in 0.004 seconds
May 07 15:25:46.081434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:46.081607 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_router_static_routes {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:46.081723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Mark Neutron-owned static routes in OVN external_ids. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:46.082796 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_router_static_routes finished in 0.001 seconds
May 07 15:25:46.082933 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:46.083120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_security_group_with_address_group {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:46.083324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Repair Address_Sets and ACLs for address group rules. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:46.098186 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_security_group_with_address_group finished in 0.015 seconds
May 07 15:25:46.098355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:46.099481 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=105503) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 07 15:25:46.824437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=12c93531-e9ce-4167-9d62-5be5296d287c, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73', 'neutron:revision_number': '8', 'neutron:router_name': 'neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7', 'neutron:subnet_ids': '6aa26064-0a62-428b-a8ed-fef52fc73984 adf79775-4dfa-4110-bca0-59be9560299c'}, gateway_chassis=[], ha_chassis_group=62868aa1-45f1-4261-a873-17ebb0e28674, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:d8:0e:a6', name='lrp-097c326d-2b42-4534-81d1-4cba26d8b34a', networks=['172.24.5.183/24', '2001:db8::1/64'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=105503) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 07 15:25:46.825702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=41ca1b19-8319-4de1-89cd-0de128f16aee, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7', 'neutron:subnet_ids': '6aa26064-0a62-428b-a8ed-fef52fc73984'}, gateway_chassis=[], ha_chassis_group=62868aa1-45f1-4261-a873-17ebb0e28674, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:d8:0e:a6', name='lrp-097c326d-2b42-4534-81d1-4cba26d8b34a', networks=['172.24.5.183/24'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={'hosting-chassis': '6c608f94-302c-45c4-8e1f-fa23d1659434'}) old= {{(pid=105503) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 07 15:25:46.827145 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 07 15:25:46.886817 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32 synchronized; highest priority chassis 6c608f94-302c-45c4-8e1f-fa23d1659434
May 07 15:25:46.887852 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '8b456c85-5e2d-47f8-9886-ebf65932cf32', 'neutron:router_id': '165b5214-095f-4c34-9f9c-9f0a42407cb7'}),), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:46.888301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, chassis=6c608f94-302c-45c4-8e1f-fa23d1659434, priority=1, columns={}) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:46.889118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.890611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 07 15:25:46.890611 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.890611 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:46.890611 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:46.890611 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.890611 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:46.890611 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.890611 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event Traceback (most recent call last):
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 89, in run
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1665, in update_router_ha_chassis_group
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2416, in link_network_ha_chassis_group
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     next(self.gen)
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     next(self.gen)
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.891931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     with self.create_transaction(
May 07 15:25:46.892932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.892932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     self.result = self.commit()
May 07 15:25:46.892932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 07 15:25:46.892932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.892932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     raise result.ex
May 07 15:25:46.892932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.892932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 07 15:25:46.892932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 07 15:25:46.892932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.892932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 07 15:25:46.892932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.892932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event 
May 07 15:25:46.924536 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32 synchronized; highest priority chassis 6c608f94-302c-45c4-8e1f-fa23d1659434
May 07 15:25:46.925283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '8b456c85-5e2d-47f8-9886-ebf65932cf32', 'neutron:router_id': '165b5214-095f-4c34-9f9c-9f0a42407cb7'}),), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:46.925732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, chassis=6c608f94-302c-45c4-8e1f-fa23d1659434, priority=1, columns={}) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:46.926037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.926855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 07 15:25:46.926855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.926855 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:46.926855 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:46.926855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.926855 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:46.926855 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.926855 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event Traceback (most recent call last):
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 81, in run
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1665, in update_router_ha_chassis_group
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2416, in link_network_ha_chassis_group
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     next(self.gen)
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     next(self.gen)
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:46.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     with self.create_transaction(
May 07 15:25:46.928853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:46.928853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     self.result = self.commit()
May 07 15:25:46.928853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 07 15:25:46.928853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:46.928853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     raise result.ex
May 07 15:25:46.928853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:46.928853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 07 15:25:46.928853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 07 15:25:46.928853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:46.928853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 07 15:25:46.928853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:46.928853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event 
May 07 15:25:50.572984 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:25:50.573587 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:50.573812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:50.575012 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:25:50.776896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:50.777268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:50.778084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:50.778084 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:50.778084 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:50.778084 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:50.778084 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:50.778084 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:50.778084 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:50.778084 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:50.778837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.204 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:50.778837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:50.778837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:50.778837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:50.778837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:50.778837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:25:50.778837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:50.778837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:50.778837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:50.778837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:50.778837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:50.778837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:50.779198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:50.779198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:50.779198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:50.779198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:50.779198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:50.779198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:50.779198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:50.779198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:50.779198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:50.779198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:50.779198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:50.779198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:50.779198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:50.779525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:50.779525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:50.779788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:50.780548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:50.781031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:50.781031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:50.781031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:51.032283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:25:51.033166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:51.033425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:51.034784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:51.035198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.036233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:51.036233 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.036233 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:51.036233 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:51.036233 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.036233 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:51.036233 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.036233 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:51.038048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.038048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:51.038048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:51.038048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:51.038048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:51.038048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:25:51.038048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:51.038048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.038048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:51.038048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:51.038048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:51.038048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.038677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:51.038677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:51.038677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:51.038677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:51.038677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:51.038677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:51.038677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:51.038677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:51.038677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.038677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:51.038677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:51.038677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.038677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:51.039275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.039275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:51.041048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:51.041854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.042671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:51.042671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.042671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:51.050118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:25:51.050536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:51.050657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:51.050788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:25:51.051258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:51.051519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.052117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:51.052117 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.052117 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:51.052117 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:51.052117 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.052117 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:51.052117 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.052117 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:25:51.052954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:51.053832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:51.054477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:25:51.054477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:25:51.054477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:25:51.054477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:51.054477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:51.054477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.054477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:51.054477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:51.054477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:51.054477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.054477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:51.054477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:51.054477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:51.055011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:51.055011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:51.055011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:51.055011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:51.055011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:51.055011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.055011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:51.055011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:51.055011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.055011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:51.055011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.055011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:51.055544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:51.056189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:51.056974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.056974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:51.056974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:51.056974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.056974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:51.056974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.056974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:51.065559 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:25:51.065993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:51.066179 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:51.066628 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:25:51.067835 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:51.067835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.068282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:51.068282 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.068282 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:51.068282 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:51.068282 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.068282 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:51.068282 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.068282 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:25:51.069540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:25:51.069961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:51.069961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:25:51.069961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.069961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:25:51.069961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:25:51.069961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.069961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:25:51.069961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.069961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:25:51.070938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.070938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:51.070938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:51.070938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:51.070938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:51.070938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:25:51.070938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:25:51.070938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:51.070938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:51.070938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.070938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:51.071507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:51.072152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.072152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:51.072152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.072152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:51.072641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:51.073477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:51.074372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.074372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:51.074372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.074372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:51.074372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:25:51.074372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:51.074372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:51.074372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:25:51.074982 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:51.075502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:51.075778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.076746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:51.076746 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.076746 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:51.076746 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:51.076746 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.076746 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:51.076746 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.076746 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:51.078063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.078063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:51.078063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:51.078063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:51.078063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:51.078063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:25:51.078063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:51.078063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.078063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:51.078063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:51.078063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:51.078063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.078554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:51.078554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:51.078554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:51.078554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:51.078554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:51.078554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:51.078554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:51.078554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:51.078554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.078554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:51.078554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:51.078554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.078554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:51.078921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.078921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:51.079705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:51.080142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:51.080658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:51.080658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:51.080658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:55.575605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:25:55.578218 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:55.578218 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:55.578218 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:25:55.631697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:55.631898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:55.632510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:55.632510 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:55.632510 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:55.632510 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:55.632510 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:55.632510 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:55.632510 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:55.632510 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:55.633433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.056 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:55.633433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:55.633433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:55.633433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:55.633433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:55.633433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:25:55.633433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:55.633433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:55.633433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:55.633433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:55.633433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:55.633433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:55.633715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:55.633715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:55.633715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:55.633715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:55.633715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:55.633715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:55.633715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:55.633715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:55.633715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:55.633715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:55.633715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:55.633715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:55.633715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:55.633972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:55.633972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:55.634164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:55.634516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:55.634907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:55.634907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:55.634907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:56.034873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:25:56.034873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:56.034873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:56.034873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:56.034873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.035570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:56.035570 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.035570 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:56.035570 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:56.035570 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.035570 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:56.035570 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.035570 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:56.036618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.036618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:56.036618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:56.036618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:56.036618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:56.036618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:25:56.036618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:56.036618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.036618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:56.036618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:56.036618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:56.036618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.037449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:56.037449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:56.037449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:56.037449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:56.037449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:56.037449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:56.037449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:56.037449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:56.037449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.037449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:56.037449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:56.037449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.037449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:56.038063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:56.038881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.039877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:56.039877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:56.039877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.039877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:56.039877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.039877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:56.050695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:25:56.051127 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:56.051267 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:56.051402 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:25:56.051923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:56.052263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.052961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:56.052961 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.052961 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:56.052961 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:56.052961 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.052961 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:56.052961 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.052961 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:25:56.053816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:56.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:56.055535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:25:56.055535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:25:56.055535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:25:56.055535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:56.055535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:56.055535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.055535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:56.055535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:56.055535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:56.055535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.055535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:56.055535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:56.055535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:56.056273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:56.056273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:56.056273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:56.056273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:56.056273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:56.056273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.056273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:56.056273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:56.056273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.056273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:56.056273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.056273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:56.056932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:56.057848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:56.058800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.058800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:56.058800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:56.058800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.058800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:56.058800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.058800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:56.065952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:25:56.066313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:56.066430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:56.066568 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:25:56.067031 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:56.067443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.068173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:56.068173 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.068173 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:56.068173 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:56.068173 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.068173 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:56.068173 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.068173 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:25:56.069054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:25:56.069791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:56.069791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:25:56.069791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.069791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:25:56.069791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:25:56.069791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.069791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:25:56.069791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.069791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:25:56.070190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.070190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:56.070190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:56.070190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:56.070190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:56.070190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:25:56.070190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:25:56.070190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:56.070190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:56.070190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.070190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:56.070766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:56.071455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:56.072125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:56.073133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:56.073133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:56.073133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:56.073133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:56.073133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.073133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:56.073133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:56.073133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.073133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:56.073133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.073133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:25:56.073916 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:25:56.074200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:25:56.074336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:25:56.074496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:25:56.074910 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:56.075305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:25:56.075491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.076225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:25:56.076225 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.076225 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:25:56.076225 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:25:56.076225 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.076225 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:25:56.076225 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.076225 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:25:56.077089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.077089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:25:56.077089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:56.077089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:25:56.077089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:25:56.077089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:25:56.077089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:56.077089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.077089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:56.077089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:56.077089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:25:56.077089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.077471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:25:56.077471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:56.077471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:25:56.077471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:56.077471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:25:56.077471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:25:56.077471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:56.077471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:25:56.077471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.077471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:25:56.077471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:25:56.077471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.077471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:25:56.078098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:25:56.078713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:25:56.079393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:25:56.079393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:25:56.079393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:25:56.079393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:25:56.079393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:25:56.079393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:00.576562 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:00.577078 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:00.577286 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:00.578424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:00.611495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:00.611662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:00.612154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:00.612154 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:00.612154 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:00.612154 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:00.612154 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:00.612154 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:00.612154 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:00.612154 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:00.613092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:00.613092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:00.613092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:00.613092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:00.613092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:00.613092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:00.613092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:00.613092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:00.613092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:00.613092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:00.613092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:00.613092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:00.613375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:00.613375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:00.613375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:00.613375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:00.613375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:00.613375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:00.613375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:00.613375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:00.613375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:00.613375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:00.613375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:00.613375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:00.613375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:00.613772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:00.613772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:00.614413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:00.614759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:00.615287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:00.615287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:00.615287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:01.033658 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:01.034056 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:01.034280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:01.035347 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:01.035665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.036517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:01.036517 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.036517 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:01.036517 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:01.036517 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.036517 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:01.036517 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.036517 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:01.038094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.038094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:01.038094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:01.038094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:01.038094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:01.038094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:01.038094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:01.038094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.038094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:01.038094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:01.038094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:01.038094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.038707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:01.038707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:01.038707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:01.038707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:01.038707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:01.038707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:01.038707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:01.038707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:01.038707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.038707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:01.038707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:01.038707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.038707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:01.039344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.039344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:01.040091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:01.040852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.041708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:01.041708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.041708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:01.051150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:01.051588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:01.051764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:01.051953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:26:01.052450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:01.052738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.053490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:01.053490 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.053490 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:01.053490 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:01.053490 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.053490 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:01.053490 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.053490 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:01.054657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:01.055382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:01.055382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:01.055382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.055382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:01.055382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:01.055382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.055382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:01.055382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.055382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:01.055943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.055943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:01.055943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:01.055943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:01.055943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:01.055943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:01.055943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:01.055943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:26:01.055943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:01.055943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:01.055943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.055943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:01.056518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:01.057112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.057112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:01.057112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.057112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:01.057588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:01.058449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:01.059282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.059282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:01.059282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:01.059282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.059282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:01.059282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.059282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:01.066336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:01.066650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:01.066831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:01.067034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:01.067627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:01.068116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.069043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:01.069043 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.069043 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:01.069043 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:01.069043 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.069043 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:01.069043 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.069043 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:01.070282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:01.070711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:01.070711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:01.070711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.070711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:01.070711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:01.070711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.070711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:01.070711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.070711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:01.071799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.071799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:01.071799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:01.071799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:01.071799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:01.071799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:01.071799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:01.071799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:01.071799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:01.071799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.071799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:01.072377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:01.073141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.073141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:01.073141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.073141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:01.073822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:01.074630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:01.075683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.075683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:01.075683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.075683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:01.075683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:01.075683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:01.075683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:01.075683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:01.076069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:01.076069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:01.076069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.076781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:01.076781 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.076781 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:01.076781 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:01.076781 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.076781 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:01.076781 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.076781 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:01.077595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.077595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:01.077595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:01.077595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:01.077595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:01.077595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:01.077595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:01.077595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.077595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:01.077595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:01.077595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:01.077595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.078172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:01.078172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:01.078172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:01.078172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:01.078172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:01.078172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:01.078172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:01.078172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:01.078172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.078172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:01.078172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:01.078172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.078172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:01.078838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:01.079570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:01.080486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:01.080486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:01.080486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:01.080486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:01.080486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:01.080486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:05.577364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:05.578179 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:05.578330 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:05.579775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:05.616853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:05.617086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:05.617786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:05.617786 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:05.617786 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:05.617786 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:05.617786 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:05.617786 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:05.617786 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:05.617786 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:05.618672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:05.618672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:05.618672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:05.618672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:05.618672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:05.618672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:05.618672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:05.618672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:05.618672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:05.618672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:05.618672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:05.618672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:05.619559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:05.619559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:05.619559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:05.619559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:05.619559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:05.619559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:05.619559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:05.619559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:05.619559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:05.619559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:05.619559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:05.619559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:05.619559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:05.620662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:05.621365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:05.622382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:05.622382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:05.622382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:05.622382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:05.622382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:05.622382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:06.034767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:06.035781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:06.035986 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:06.036916 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:06.037108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.037755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:06.037755 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.037755 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:06.037755 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:06.037755 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.037755 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:06.037755 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.037755 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:06.039354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.039354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:06.039354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:06.039354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:06.039354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:06.039354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:06.039354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:06.039354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.039354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:06.039354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:06.039354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:06.039354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.040039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:06.040039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:06.040039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:06.040039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:06.040039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:06.040039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:06.040039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:06.040039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:06.040039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.040039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:06.040039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:06.040039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.040039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:06.040734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:06.041566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.042537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:06.042537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:06.042537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.042537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:06.042537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.042537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:06.051655 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:06.051942 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:06.052110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:06.052320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:26:06.052789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:06.053039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.053711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:06.053711 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.053711 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:06.053711 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:06.053711 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.053711 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:06.053711 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.053711 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:06.054900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:06.055679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:06.055679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:06.055679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.055679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:06.055679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:06.055679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.055679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:06.055679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.055679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:06.056379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.056379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:06.056379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:06.056379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:06.056379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:06.056379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:06.056379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:06.056379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:26:06.056379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:06.056379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:06.056379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.056379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:06.056983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:06.057587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.057587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:06.057587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.057587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:06.057963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:06.058745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:06.059577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.059577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:06.059577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:06.059577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.059577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:06.059577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.059577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:06.066673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:06.067004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:06.067185 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:06.067487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:06.068039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:06.068553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.069547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:06.069547 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.069547 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:06.069547 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:06.069547 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.069547 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:06.069547 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.069547 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:06.071141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:06.071852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:06.071852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:06.071852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.071852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:06.071852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:06.071852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.071852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:06.071852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.071852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:06.073586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.073586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:06.073586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:06.073586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:06.073586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:06.073586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:06.073586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:06.073586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:06.073586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:06.073586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.073586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:06.074155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:06.074824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.074824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:06.074824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.074824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:06.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:06.076747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:06.077613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.077613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:06.077613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.077613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:06.077613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:06.077613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:06.077613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:06.077613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:06.078025 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:06.078025 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:06.078025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.078025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:06.078025 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.078025 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:06.078025 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:06.078025 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.078025 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:06.078025 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.078025 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:06.078660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.078660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:06.078660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:06.078660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:06.078660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:06.078660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:06.078660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:06.078660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.078660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:06.078660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:06.078660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:06.078660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.079035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:06.079035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:06.079035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:06.079035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:06.079035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:06.079035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:06.079035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:06.079035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:06.079035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.079035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:06.079035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:06.079035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.079035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:06.079445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:06.079899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:06.080485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:06.080485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:06.080485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:06.080485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:06.080485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:06.080485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:10.577932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:10.578445 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:10.578445 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:10.579279 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:10.609157 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:10.609570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:10.610480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:10.610480 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:10.610480 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:10.610480 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:10.610480 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:10.610480 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:10.610480 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:10.610480 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:10.611902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:10.611902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:10.611902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:10.611902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:10.611902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:10.611902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:10.611902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:10.611902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:10.611902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:10.611902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:10.611902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:10.611902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:10.612274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:10.612274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:10.612274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:10.612274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:10.612274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:10.612274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:10.612274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:10.612274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:10.612274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:10.612274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:10.612274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:10.612274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:10.612274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:10.612604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:10.612604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:10.613906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:10.614357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:10.614861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:10.614861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:10.614861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:11.035769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:11.036285 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:11.036421 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:11.037342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:11.037643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.038446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:11.038446 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.038446 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:11.038446 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:11.038446 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.038446 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:11.038446 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.038446 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:11.039562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.039562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:11.039562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:11.039562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:11.039562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:11.039562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:11.039562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:11.039562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.039562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:11.039562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:11.039562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:11.039562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.040174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:11.040174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:11.040174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:11.040174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:11.040174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:11.040174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:11.040174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:11.040174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:11.040174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.040174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:11.040174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:11.040174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.040174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:11.040842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:11.041618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.042990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:11.042990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:11.042990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.042990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:11.042990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.042990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:11.052077 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:11.052367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:11.052479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:11.052601 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:26:11.053075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:11.053408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.054110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:11.054110 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.054110 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:11.054110 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:11.054110 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.054110 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:11.054110 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.054110 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:11.054969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:11.055657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:11.055657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:11.055657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.055657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:11.055657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:11.055657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.055657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:11.055657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.055657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:11.056080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.056080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:11.056080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:11.056080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:11.056080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:11.056080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:11.056080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:11.056080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:26:11.056080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:11.056080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:11.056080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.056080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:11.056695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:11.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:11.057961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:11.058790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:11.058790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:11.058790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:11.058790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:11.058790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:11.058790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.058790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:11.058790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:11.058790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.058790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:11.058790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.058790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:11.067116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:11.067414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:11.067544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:11.067685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:11.068125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:11.068521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.069223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:11.069223 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.069223 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:11.069223 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:11.069223 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.069223 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:11.069223 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.069223 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:11.070057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:11.070471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:11.070471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:11.070471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.070471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:11.070471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:11.070471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.070471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:11.070471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.070471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:11.071041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.071041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:11.071041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:11.071041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:11.071041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:11.071041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:11.071041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:11.071041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:11.071041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:11.071041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.071041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:11.071352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:11.071705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.071705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:11.071705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.071705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:11.072269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:11.072721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:11.073223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.073223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:11.073223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.073223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:11.076592 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:11.076997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:11.077191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:11.077473 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:11.077900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:11.078257 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:11.078437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.079184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:11.079184 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.079184 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:11.079184 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:11.079184 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.079184 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:11.079184 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.079184 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:11.080666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.080666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:11.080666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:11.080666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:11.080666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:11.080666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:11.080666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:11.080666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.080666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:11.080666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:11.080666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:11.080666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.081028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:11.081028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:11.081028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:11.081028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:11.081028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:11.081028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:11.081028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:11.081028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:11.081028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.081028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:11.081028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:11.081028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.081028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:11.081382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.081382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:11.082299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:11.082735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:11.083244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:11.083244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:11.083244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:15.578511 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:15.579088 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:15.579319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:15.580554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:15.614559 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:15.614751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:15.615499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:15.615499 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:15.615499 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:15.615499 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:15.615499 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:15.615499 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:15.615499 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:15.615499 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:15.616545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:15.616545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:15.616545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:15.616545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:15.616545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:15.616545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:15.616545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:15.616545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:15.616545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:15.616545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:15.616545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:15.616545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:15.617100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:15.617100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:15.617100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:15.617100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:15.617100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:15.617100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:15.617100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:15.617100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:15.617100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:15.617100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:15.617100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:15.617100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:15.617100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:15.617693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:15.618373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:15.619102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:15.619102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:15.619102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:15.619102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:15.619102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:15.619102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:16.110777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:16.110777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:16.110777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:16.110777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:16.110777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.110777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:16.110777 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.110777 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:16.110777 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:16.112041 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.112041 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:16.112041 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.112041 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:16.112041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.112041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:16.112041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:16.112041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:16.112041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:16.112041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:16.112041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:16.112041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.112041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:16.113725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:16.115555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:16.117098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:16.119108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:16.119108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:16.119108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.119108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:16.119108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:16.119108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.119108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:16.119108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.119108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:16.119108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:16.119108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:16.119108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:16.120400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:26:16.120400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:16.120400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.120400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:16.120400 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.120400 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:16.120400 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:16.120400 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.120400 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:16.120400 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.120400 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:16.120400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.120400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.121315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:16.122071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:16.122071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.122071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:16.122071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.122071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:16.122071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.122071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:16.122071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:16.122071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:16.122071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:16.122071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:16.122071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:16.122071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:26:16.122928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:16.122928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:16.122928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.122928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:16.122928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:16.122928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:16.122928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.122928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:16.122928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:16.122928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:16.122928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:16.122928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:16.122928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:16.123520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:16.123520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:16.123520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.123520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:16.123520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:16.123520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.123520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:16.123520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.123520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:16.123520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.123520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:16.123520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.124092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.125003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:16.125665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:16.125665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:16.125665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:16.125665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:16.125665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:16.125665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.125855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:16.126254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.126551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:16.126551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:16.126551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:16.126551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:16.126551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:16.126551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:16.126551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:16.126551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:16.126551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.126551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:16.126551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:16.126551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:16.126551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.126846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:16.126846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:16.126846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:16.126846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:16.126846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:16.126846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:16.126846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:16.126846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:16.126846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.126846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:16.126846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:16.126846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.126846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:16.127143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:16.128477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:16.129245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:16.129245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.129245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:16.129245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:16.129245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.129245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:16.129245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.129245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:16.129245 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:16.129245 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:16.129245 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:16.129695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:16.129695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:16.129695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:16.129695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.129695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:16.129695 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.129695 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:16.129695 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:16.129695 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.129695 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:16.129695 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.129695 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:16.130171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.130171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:16.130171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:16.130171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:16.130171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:16.130171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:16.130171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:16.130171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.130171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:16.130171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:16.130171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:16.130171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.130450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:16.130450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:16.130450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:16.130450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:16.130450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:16.130450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:16.130450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:16.130450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:16.130450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.130450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:16.130450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:16.130450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.130450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:16.130676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:16.131013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:16.131352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:16.131352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:16.131352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:16.131352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:16.131352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:16.131352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:20.579136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:20.580521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:20.580786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:20.582393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:20.623044 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:20.623295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:20.625016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:20.625016 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:20.625016 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:20.625016 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:20.625016 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:20.625016 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:20.625016 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:20.625016 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:20.626736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:20.626736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:20.626736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:20.626736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:20.626736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:20.626736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:20.626736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:20.626736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:20.626736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:20.626736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:20.626736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:20.626736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:20.627134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:20.627134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:20.627134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:20.627134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:20.627134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:20.627134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:20.627134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:20.627134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:20.627134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:20.627134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:20.627134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:20.627134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:20.627134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:20.627550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:20.627550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:20.628055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:20.628570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:20.629097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:20.629097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:20.629097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:21.042815 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:21.043076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:21.043122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:21.044002 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:21.044362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.045168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:21.045168 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.045168 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:21.045168 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:21.045168 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.045168 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:21.045168 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.045168 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:21.046190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.046190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:21.046190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:21.046190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:21.046190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:21.046190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:21.046190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:21.046190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.046190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:21.046190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:21.046190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:21.046190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.046499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:21.046499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:21.046499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:21.046499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:21.046499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:21.046499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:21.046499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:21.046499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:21.046499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.046499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:21.046499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:21.046499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.046499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:21.046771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.046771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:21.047338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:21.047668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.048112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:21.048112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.048112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:21.053091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:21.053793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:21.053876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:21.053963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:26:21.054435 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:21.054721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.056075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:21.056075 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.056075 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:21.056075 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:21.056075 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.056075 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:21.056075 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.056075 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:21.056636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:21.057042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:21.057042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:21.057042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.057042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:21.057042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:21.057042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.057042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:21.057042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.057042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:21.057407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.057407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:21.057407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:21.057407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:21.057407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:21.057407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:21.057407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:21.057407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:26:21.057407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:21.057407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:21.057407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.057407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:21.057734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:21.058095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:21.058528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:21.059113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:21.059113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:21.059113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:21.059113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:21.059113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:21.059113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.059113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:21.059113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:21.059113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.059113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:21.059113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.059113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:21.069220 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:21.069220 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:21.069220 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:21.069220 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:21.069220 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:21.069220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.072657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:21.074974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.076851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:21.076851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:21.076851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:21.076851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:21.076851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:21.076851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:21.076851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:21.076851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:21.076851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.076851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:21.076851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:21.076851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:21.076851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.078085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:21.078085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:21.078085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:21.078085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:21.078085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:21.078085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:21.078085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:21.078085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:21.078085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.078085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:21.078085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:21.078085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.078085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:21.079632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:21.082231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:21.084647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:21.084647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.084647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:21.084647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:21.084647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.084647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:21.084647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.084647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:21.084647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:21.084647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:21.084647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:21.086616 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:21.086616 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:21.086616 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:21.086616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.086616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:21.086616 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.086616 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:21.086616 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:21.086616 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.086616 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:21.086616 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.086616 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:21.087801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.087801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:21.087801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:21.087801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:21.087801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:21.087801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:21.087801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:21.087801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.087801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:21.087801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:21.087801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:21.087801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.088913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:21.088913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:21.088913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:21.088913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:21.088913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:21.088913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:21.088913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:21.088913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:21.088913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.088913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:21.088913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:21.088913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.088913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:21.090619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:21.092265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:21.093949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:21.093949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:21.093949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:21.093949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:21.093949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:21.093949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:25.580559 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:25.581435 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:25.581435 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:25.582350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:25.610149 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:25.610657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:25.611618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:25.611618 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:25.611618 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:25.611618 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:25.611618 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:25.611618 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:25.611618 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:25.611618 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:25.612617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:25.612617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:25.612617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:25.612617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:25.612617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:25.612617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:25.612617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:25.612617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:25.612617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:25.612617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:25.612617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:25.612617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:25.612992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:25.612992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:25.612992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:25.612992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:25.612992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:25.612992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:25.612992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:25.612992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:25.612992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:25.612992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:25.612992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:25.612992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:25.612992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:25.613414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:25.613414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:25.613669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:25.614051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:25.614434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:25.614434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:25.614434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:26.043340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:26.044093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:26.044373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:26.045032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:26.045220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.045864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:26.045864 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.045864 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:26.045864 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:26.045864 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.045864 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:26.045864 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.045864 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:26.047469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.047469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:26.047469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:26.047469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:26.047469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:26.047469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:26.047469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:26.047469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.047469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:26.047469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:26.047469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:26.047469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.048098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:26.048098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:26.048098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:26.048098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:26.048098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:26.048098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:26.048098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:26.048098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:26.048098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.048098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:26.048098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:26.048098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.048098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:26.048715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.048715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:26.049234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:26.049971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.050807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:26.050807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.050807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:26.054037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:26.054366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:26.054500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:26.054668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:26:26.055078 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:26.055301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.055853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:26.055853 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.055853 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:26.055853 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:26.055853 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.055853 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:26.055853 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.055853 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:26.056781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:26.057256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:26.057256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:26.057256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.057256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:26.057256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:26.057256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.057256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:26.057256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.057256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:26.057906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.057906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:26.057906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:26.057906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:26.057906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:26.057906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:26.057906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:26.057906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:26:26.057906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:26.057906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:26.057906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.057906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:26.058296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:26.058722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.058722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:26.058722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.058722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:26.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:26.059800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:26.060591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.060591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:26.060591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:26.060591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.060591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:26.060591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.060591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:26.101307 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:26.101307 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:26.102046 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:26.102046 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:26.102046 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:26.102541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:26.102920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.103750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:26.103750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.103750 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:26.103750 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:26.103750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.103750 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:26.103750 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.103750 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:26.104748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:26.105236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:26.105236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:26.105236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.105236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:26.105236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:26.105236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.105236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:26.105236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.105236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:26.105708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.105708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:26.105708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:26.105708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:26.105708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:26.105708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:26.105708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:26.105708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:26.105708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:26.105708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.105708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:26.106043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:26.106468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.106468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:26.106468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.106468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:26.106886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:26.107445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:26.107969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.107969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:26.107969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.107969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:26.107969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:26.107969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:26.107969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:26.107969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:26.108278 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:26.108278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.108647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:26.108647 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.108647 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:26.108647 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:26.108647 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.108647 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:26.108647 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.108647 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:26.109478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.109478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:26.109478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:26.109478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:26.109478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:26.109478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:26.109478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:26.109478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.109478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:26.109478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:26.109478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:26.109478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.109837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:26.109837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:26.109837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:26.109837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:26.109837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:26.109837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:26.109837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:26.109837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:26.109837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.109837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:26.109837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:26.109837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.109837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:26.110184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.110184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:26.110497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:26.110949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:26.111513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:26.111513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:26.111513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:30.581165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:30.581914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:30.581914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:30.583167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:30.616531 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:30.616967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:30.617820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:30.617820 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:30.617820 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:30.617820 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:30.617820 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:30.617820 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:30.617820 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:30.617820 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:30.618663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:30.618663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:30.618663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:30.618663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:30.618663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:30.618663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:30.618663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:30.618663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:30.618663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:30.618663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:30.618663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:30.618663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:30.619037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:30.619037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:30.619037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:30.619037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:30.619037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:30.619037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:30.619037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:30.619037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:30.619037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:30.619037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:30.619037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:30.619037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:30.619037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:30.619455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:30.619455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:30.619693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:30.620038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:30.620442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:30.620442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:30.620442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:31.044374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:31.044846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:31.045036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:31.045843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:31.046120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.046941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:31.046941 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.046941 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:31.046941 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:31.046941 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.046941 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:31.046941 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.046941 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:31.048508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.048508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:31.048508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:31.048508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:31.048508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:31.048508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:31.048508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:31.048508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.048508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:31.048508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:31.048508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:31.048508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.049015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:31.049015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:31.049015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:31.049015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:31.049015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:31.049015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:31.049015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:31.049015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:31.049015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.049015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:31.049015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:31.049015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.049015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:31.049582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.049582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:31.050130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:31.051656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.052730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:31.052730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.052730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:31.054461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:31.054763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:31.054886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:31.054991 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:26:31.055524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:31.055811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.056480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:31.056480 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.056480 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:31.056480 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:31.056480 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.056480 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:31.056480 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.056480 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:31.057226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:31.057616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:31.057616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:31.057616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.057616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:31.057616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:31.057616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.057616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:31.057616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.057616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:31.058120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.058120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:31.058120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:31.058120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:31.058120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:31.058120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:31.058120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:31.058120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:26:31.058120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:31.058120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:31.058120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.058120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:31.058813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:31.059547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:31.059894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:31.060374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:31.060374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:31.060374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:31.060374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:31.060374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:31.060374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.060374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:31.060374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:31.060374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.060374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:31.060374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.060374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:31.101699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:31.102089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:31.102293 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:31.102507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:31.102956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:31.103375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.104034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:31.104034 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.104034 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:31.104034 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:31.104034 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.104034 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:31.104034 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.104034 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:31.105188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:31.106901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:31.107596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:31.107596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:31.107596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:31.107596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:31.107596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.107596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:31.107596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:31.107596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:31.107596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.107596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:31.107596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:31.107596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:31.107596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:31.109367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:31.109367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:31.109367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:31.109367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:31.109367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.109367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:31.109367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:31.109367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.109367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:31.109367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.109367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:31.109367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:31.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.112177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:31.113733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:31.113733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:31.113733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:31.113733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:31.113733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:31.113733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:31.113990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:31.114286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:31.114542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.114542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:31.114542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:31.114542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.114542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:31.114542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.114542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:31.114542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.114542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:31.114542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:31.114542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:31.114542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:31.114542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:31.114779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:31.115127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:31.115127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:31.115127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:31.115127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:31.115127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:31.115127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:31.115127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:31.115127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:31.115127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:31.115127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:31.115127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:31.115127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:35.581851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:35.585399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:35.585399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:35.585399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:35.587236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=560,neutron:INSERT=1,neutron:UPDATE=2,neutron:DELETE=1 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:26:35.622903 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:35.623729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:35.624901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:35.624901 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:35.624901 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:35.624901 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:35.624901 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:35.624901 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:35.624901 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:35.624901 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:35.626260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:35.626260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:35.626260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:35.626260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:35.626260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:35.626260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:35.626260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:35.626260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:35.626260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:35.626260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:35.626260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:35.626260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:35.626654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:35.626654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:35.626654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:35.626654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:35.626654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:35.626654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:35.626654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:35.626654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:35.626654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:35.626654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:35.626654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:35.626654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:35.626654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:35.627026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:35.627026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:35.628287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:35.628763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:35.629372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:35.629372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:35.629372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:36.045155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:36.046150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:36.046150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:36.046610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:36.047075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.049314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:36.049314 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.049314 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:36.049314 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:36.049314 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.049314 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:36.049314 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.049314 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:36.051140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.051140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:36.051140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:36.051140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:36.051140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:36.051140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:36.051140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:36.051140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.051140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:36.051140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:36.051140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:36.051140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.051541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:36.051541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:36.051541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:36.051541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:36.051541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:36.051541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:36.051541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:36.051541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:36.051541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.051541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:36.051541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:36.051541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.051541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:36.051978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:36.052446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.053067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:36.053067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:36.053067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.053067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:36.053067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.053067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:36.056555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:36.056555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:36.056555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:36.056555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:26:36.056555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:36.056555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.058546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:36.059127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.059624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:36.059624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:36.059624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:36.059624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:36.059624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:36.059624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:36.059624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:26:36.059624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:36.059624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:36.059624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.059624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:36.059624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:36.059624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.059888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:36.060173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:36.061787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:36.063052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:36.063052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:36.063052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.063052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:36.063052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:36.063052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.063052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:36.063052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.063052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:36.102177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:36.102669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:36.102784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:36.102914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:36.103444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:36.103864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.104633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:36.104633 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.104633 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:36.104633 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:36.104633 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.104633 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:36.104633 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.104633 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:36.105482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:36.106157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:36.106157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:36.106157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.106157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:36.106157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:36.106157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.106157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:36.106157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.106157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:36.106565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.106565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:36.106565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:36.106565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:36.106565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:36.106565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:36.106565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:36.106565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:36.106565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:36.106565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.106565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:36.107042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:36.107636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:36.108273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:36.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:36.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:36.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:36.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:36.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:36.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:36.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:36.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:36.109659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:36.109659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:36.110357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:36.110357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:36.110357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:36.110700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:36.110884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.111592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:36.111592 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.111592 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:36.111592 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:36.111592 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.111592 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:36.111592 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.111592 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:36.112976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.112976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:36.112976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:36.112976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:36.112976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:36.112976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:36.112976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:36.112976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.112976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:36.112976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:36.112976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:36.112976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.113397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:36.113397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:36.113397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:36.113397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:36.113397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:36.113397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:36.113397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:36.113397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:36.113397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.113397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:36.113397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:36.113397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.113397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:36.113768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:36.114221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:36.114716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:36.114716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:36.114716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:36.114716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:36.114716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:36.114716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:40.583165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:40.584107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:40.584107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:40.585259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:40.615794 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:40.616192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:40.617095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:40.617095 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:40.617095 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:40.617095 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:40.617095 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:40.617095 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:40.617095 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:40.617095 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:40.618229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:40.618229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:40.618229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:40.618229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:40.618229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:40.618229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:40.618229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:40.618229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:40.618229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:40.618229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:40.618229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:40.618229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:40.618822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:40.618822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:40.618822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:40.618822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:40.618822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:40.618822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:40.618822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:40.618822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:40.618822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:40.618822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:40.618822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:40.618822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:40.618822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:40.619436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:40.620121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:40.620977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:40.620977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:40.620977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:40.620977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:40.620977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:40.620977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:41.045694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:41.046162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:41.046382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:41.049095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:41.049535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.050524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:41.050524 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.050524 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:41.050524 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:41.050524 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.050524 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:41.050524 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.050524 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:41.052089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.052089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:41.052089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:41.052089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:41.052089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:41.052089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:41.052089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:41.052089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.052089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:41.052089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:41.052089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:41.052089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.053640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:41.053640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:41.053640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:41.053640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:41.053640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:41.053640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:41.053640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:41.053640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:41.053640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.053640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:41.053640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:41.053640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.053640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:41.054505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:41.055350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.055833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:41.055833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:41.055833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.055833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:41.055833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.055833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:41.055833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:41.055833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:41.055833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:41.055833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:26:41.056380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:41.056632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.057112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:41.057112 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.057112 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:41.057112 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:41.057112 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.057112 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:41.057112 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.057112 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:41.058033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:41.058885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:41.059674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:41.059674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:41.059674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:26:41.059674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:41.059674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:41.059674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.059674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:41.059674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:41.059674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:41.059674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.059674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:41.059674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:41.059674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:41.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:41.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:41.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:41.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:41.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:41.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:41.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:41.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:41.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:41.060382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:41.060821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:41.061329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.061329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:41.061329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:41.061329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.061329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:41.061329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.061329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:41.102788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:41.103180 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:41.103369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:41.103577 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:41.104564 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:41.104908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.105688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:41.105688 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.105688 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:41.105688 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:41.105688 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.105688 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:41.105688 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.105688 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:41.106960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:41.107645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:41.107645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:41.107645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.107645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:41.107645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:41.107645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.107645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:41.107645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.107645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:41.108919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.108919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:41.108919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:41.108919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:41.108919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:41.108919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:41.108919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:41.108919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:41.108919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:41.108919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.108919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:41.109459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:41.110006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.110006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:41.110006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.110006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:41.111671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:41.112475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:41.113239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.113239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:41.113239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.113239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:41.113239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:41.113239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:41.113239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:41.113239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:41.113614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:41.113614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:41.114301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.114301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:41.114301 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.114301 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:41.114301 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:41.114301 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.114301 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:41.114301 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.114301 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:41.115470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.115470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:41.115470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:41.115470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:41.115470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:41.115470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:41.115470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:41.115470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.115470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:41.115470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:41.115470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:41.115470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.116539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:41.116539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:41.116539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:41.116539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:41.116539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:41.116539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:41.116539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:41.116539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:41.116539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.116539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:41.116539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:41.116539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.116539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:41.117289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:41.118473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:41.119806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:41.119806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:41.119806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:41.119806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:41.119806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:41.119806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:45.584095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:45.648518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:45.648518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:45.648518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:45.648518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:45.648518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:45.648518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:45.648518 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:45.648518 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:45.648518 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:45.649331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:45.650155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:45.650973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:45.651807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:45.652489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:45.652489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:45.652489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:45.652489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:45.652489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:45.652489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:46.046280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:46.046865 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:46.047073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:46.048038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:46.048385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.049259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:46.049259 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.049259 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:46.049259 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:46.049259 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.049259 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:46.049259 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.049259 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:46.050887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.050887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:46.050887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:46.050887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:46.050887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:46.050887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:46.050887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:46.050887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.050887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:46.050887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:46.050887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:46.050887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.052342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:46.052342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:46.052342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:46.052342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:46.052342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:46.052342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:46.052342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:46.052342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:46.052342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.052342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:46.052342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:46.052342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.052342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:46.053628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:46.055006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.057148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:46.057148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:46.057148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.057148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:46.057148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.057148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:46.057148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:46.057148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:46.057148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:46.057148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:26:46.057726 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:46.057726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.058110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:46.058110 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.058110 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:46.058110 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:46.058110 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.058110 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:46.058110 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.058110 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:46.058856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:46.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:46.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:46.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:46.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:46.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:46.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.059322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:46.059909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.059909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:46.059909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:46.059909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:46.059909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:46.059909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:46.059909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:46.059909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:26:46.059909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:46.059909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:46.059909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.059909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:46.060289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:46.060628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.060628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:46.060628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.060628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:46.061110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:46.061712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:46.062177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.062177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:46.062177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:46.062177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.062177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:46.062177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.062177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:46.103399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:46.103897 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:46.104115 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:46.104381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:46.104895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:46.105286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.106032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:46.106032 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.106032 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:46.106032 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:46.106032 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.106032 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:46.106032 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.106032 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:46.107343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:46.108308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:46.108308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:46.108308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.108308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:46.108308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:46.108308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.108308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:46.108308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.108308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:46.108777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.108777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:46.108777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:46.108777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:46.108777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:46.108777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:46.108777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:46.108777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:46.108777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:46.108777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.108777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:46.109267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:46.109631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.109631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:46.109631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.109631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:46.110499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:46.110975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:46.111479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.111479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:46.111479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.111479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:46.112345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:46.112732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:46.112917 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:46.113077 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:46.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:46.113844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:46.114019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.114685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:46.114685 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.114685 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:46.114685 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:46.114685 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.114685 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:46.114685 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.114685 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:46.115494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.115494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:46.115494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:46.115494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:46.115494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:46.115494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:46.115494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:46.115494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.115494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:46.115494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:46.115494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:46.115494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.116160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:46.116160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:46.116160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:46.116160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:46.116160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:46.116160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:46.116160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:46.116160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:46.116160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.116160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:46.116160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:46.116160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.116160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:46.116647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:46.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:46.117888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:46.117888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:46.117888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:46.117888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:46.117888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:46.117888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:50.584683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:50.586496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:50.586496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:50.586662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:50.619655 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:50.620020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:50.620893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:50.620893 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:50.620893 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:50.620893 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:50.620893 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:50.620893 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:50.620893 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:50.620893 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:50.621710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:50.621710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:50.621710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:50.621710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:50.621710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:50.621710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:50.621710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:50.621710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:50.621710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:50.621710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:50.621710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:50.621710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:50.622078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:50.622078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:50.622078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:50.622078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:50.622078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:50.622078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:50.622078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:50.622078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:50.622078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:50.622078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:50.622078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:50.622078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:50.622078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:50.622503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:50.622503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:50.622580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:50.622959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:50.623442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:50.623442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:50.623442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:51.047045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:51.047572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:51.047794 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:51.048778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:51.049117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.050013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:51.050013 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.050013 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:51.050013 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:51.050013 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.050013 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:51.050013 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.050013 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:51.051628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.051628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:51.051628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:51.051628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:51.051628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:51.051628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:51.051628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:51.051628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.051628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:51.051628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:51.051628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:51.051628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.052360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:51.052360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:51.052360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:51.052360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:51.052360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:51.052360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:51.052360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:51.052360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:51.052360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.052360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:51.052360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:51.052360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.052360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:51.053183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.053183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:51.053351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:51.054074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.055112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:51.055112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.055112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:51.056148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:51.056408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:51.056510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:51.056680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:26:51.057144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:51.057472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.058162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:51.058162 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.058162 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:51.058162 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:51.058162 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.058162 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:51.058162 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.058162 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:51.058951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:51.059469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:51.059469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:51.059469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.059469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:51.059469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:51.059469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.059469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:51.059469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.059469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:51.059901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.059901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:51.059901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:51.059901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:51.059901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:51.059901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:51.059901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:51.059901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:26:51.059901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:51.059901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:51.059901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.059901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:51.060323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:51.060832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.060832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:51.060832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.060832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:51.061091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:51.061576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:51.062127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.062127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:51.062127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:51.062127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.062127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:51.062127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.062127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:51.104043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:51.104596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:51.104794 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:51.104995 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:51.105596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:51.105982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.107111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:51.107111 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.107111 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:51.107111 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:51.107111 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.107111 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:51.107111 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.107111 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:51.108353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:51.109064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:51.109064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:51.109064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.109064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:51.109064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:51.109064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.109064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:51.109064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.109064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:51.109786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.109786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:51.109786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:51.109786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:51.109786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:51.109786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:51.109786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:51.109786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:51.109786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:51.109786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.109786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:51.110380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:51.111041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.111041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:51.111041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.111041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:51.111831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:51.112640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:51.113456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.113456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:51.113456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.113456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:51.113456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:51.113456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:51.113456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:51.113456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:51.113858 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:51.114197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:51.114392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.115078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:51.115078 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.115078 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:51.115078 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:51.115078 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.115078 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:51.115078 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.115078 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:51.115948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.115948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:51.115948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:51.115948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:51.115948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:51.115948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:51.115948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:51.115948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.115948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:51.115948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:51.115948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:51.115948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.116391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:51.116391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:51.116391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:51.116391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:51.116391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:51.116391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:51.116391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:51.116391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:51.116391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.116391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:51.116391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:51.116391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.116391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:51.116814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.116814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:51.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:51.117517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:51.118132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:51.118132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:51.118132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:55.585343 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:55.585896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:55.586123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:55.587377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:55.617474 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:55.617474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:55.617671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:55.617671 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:55.617671 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:55.617671 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:55.617671 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:55.617671 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:55.617671 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:55.617671 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:55.618434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:55.618434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:55.618434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:55.618434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:55.618434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:55.618434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:55.618434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:55.618434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:55.618434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:55.618434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:55.618434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:55.618434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:55.618792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:55.618792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:55.618792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:55.618792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:55.618792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:55.618792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:55.618792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:55.618792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:55.618792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:55.618792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:55.618792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:55.618792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:55.618792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:55.619122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:55.619122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:55.619181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:55.619666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:55.620653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:55.620653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:55.620653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:56.047479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:56.048064 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:56.048310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:56.050318 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:56.050318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.050566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:56.050566 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.050566 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:56.050566 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:56.050566 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.050566 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:56.050566 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.050566 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:56.052088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.052088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:56.052088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:56.052088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:56.052088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:56.052088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:56.052088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:56.052088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.052088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:56.052088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:56.052088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:56.052088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.052667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:56.052667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:56.052667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:56.052667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:56.052667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:56.052667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:56.052667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:56.052667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:56.052667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.052667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:56.052667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:56.052667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.052667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:56.053285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.053285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:56.053837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:56.054572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.055491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:56.055491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.055491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:56.056543 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:56.057015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:56.057225 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:56.057367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:26:56.057826 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:56.058280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.058940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:56.058940 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.058940 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:56.058940 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:56.058940 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.058940 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:56.058940 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.058940 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:56.059683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:56.060079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:56.060079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:56.060079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.060079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:56.060079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:56.060079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.060079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:56.060079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.060079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:56.060517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.060517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:56.060517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:56.060517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:56.060517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:56.060517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:56.060517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:56.060517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:26:56.060517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:56.060517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:56.060517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.060517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:56.060869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:56.061245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.061245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:56.061245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.061245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:56.061545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:56.061983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:56.062472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.062472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:56.062472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:56.062472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.062472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:56.062472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.062472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:56.104534 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:56.105178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:56.105479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:56.105764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:56.107368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:56.107368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.107690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:56.107690 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.107690 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:56.107690 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:56.107690 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.107690 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:56.107690 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.107690 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:26:56.109547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:26:56.110265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:56.110265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:26:56.110265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.110265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:26:56.110265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:26:56.110265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.110265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:26:56.110265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.110265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:26:56.111802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.111802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:56.111802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:56.111802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:56.111802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:56.111802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:56.111802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:56.111802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:56.111802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:56.111802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.111802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:56.112280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:56.112725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.112725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:56.112725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.112725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:56.114093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:56.114565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:56.115059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.115059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:56.115059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.115059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:26:56.115059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:26:56.115059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:26:56.115059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:26:56.115059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:26:56.115566 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:56.115938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:26:56.116194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.116907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:26:56.116907 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.116907 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:26:56.116907 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:26:56.116907 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.116907 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:26:56.116907 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.116907 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:26:56.117531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.117531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:26:56.117531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:56.117531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:26:56.117531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:26:56.117531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:56.117531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:56.117531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.117531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:56.117531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:56.117531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:26:56.117531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.117924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:26:56.117924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:56.117924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:26:56.117924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:56.117924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:26:56.117924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:26:56.117924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:56.117924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:26:56.117924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.117924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:26:56.117924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:26:56.117924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.117924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:26:56.118309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:26:56.118761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:26:56.119324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:26:56.119324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:26:56.119324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:26:56.119324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:26:56.119324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:26:56.119324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:00.586104 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:00.587315 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:00.587315 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:00.588750 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:00.620697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:00.621056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:00.621855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:00.621855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:00.621855 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:00.621855 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:00.621855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:00.621855 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:00.621855 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:00.621855 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:00.622627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:00.622627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:00.622627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:00.622627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:00.622627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:00.622627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:00.622627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:00.622627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:00.622627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:00.622627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:00.622627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:00.622627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:00.623239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:00.623239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:00.623239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:00.623239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:00.623239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:00.623239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:00.623239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:00.623239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:00.623239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:00.623239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:00.623239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:00.623239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:00.623239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:00.623878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:00.624606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:00.625504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:00.625504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:00.625504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:00.625504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:00.625504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:00.625504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:01.048286 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:01.048932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:01.049136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:01.050155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:01.050511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.051425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:01.051425 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.051425 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:01.051425 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:01.051425 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.051425 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:01.051425 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.051425 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:01.052967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.052967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:01.052967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:01.052967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:01.052967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:01.052967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:01.052967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:01.052967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.052967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:01.052967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:01.052967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:01.052967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.053565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:01.053565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:01.053565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:01.053565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:01.053565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:01.053565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:01.053565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:01.053565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:01.053565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.053565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:01.053565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:01.053565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.053565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:01.054136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.054136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:01.054659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:01.055392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.056388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:01.056388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.056388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:01.057122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:01.057518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:01.057697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:01.057950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:27:01.058452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:01.058732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.059391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:01.059391 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.059391 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:01.059391 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:01.059391 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.059391 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:01.059391 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.059391 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:01.060198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:01.060658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:01.060658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:01.060658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.060658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:01.060658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:01.060658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.060658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:01.060658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.060658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:01.061031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.061031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:01.061031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:01.061031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:01.061031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:01.061031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:01.061031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:01.061031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:27:01.061031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:01.061031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:01.061031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.061031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:01.061411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:01.061803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.061803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:01.061803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.061803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:01.062100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:01.062583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:01.063097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.063097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:01.063097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:01.063097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.063097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:01.063097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.063097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:01.105232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:01.105671 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:01.105905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:01.106108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:01.106681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:01.107060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.107793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:01.107793 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.107793 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:01.107793 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:01.107793 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.107793 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:01.107793 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.107793 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:01.109078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:01.109797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:01.109797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:01.109797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.109797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:01.109797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:01.109797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.109797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:01.109797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.109797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:01.110822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.110822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:01.110822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:01.110822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:01.110822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:01.110822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:01.110822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:01.110822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:01.110822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:01.110822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.110822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:01.111380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:01.111966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.111966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:01.111966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.111966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:01.112596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:01.113408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:01.114299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.114299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:01.114299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.114299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:01.114887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:01.115284 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:01.115466 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:01.115718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:01.116024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:01.116260 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:01.116378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.116828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:01.116828 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.116828 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:01.116828 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:01.116828 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.116828 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:01.116828 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.116828 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:01.117623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.117623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:01.117623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:01.117623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:01.117623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:01.117623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:01.117623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:01.117623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.117623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:01.117623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:01.117623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:01.117623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.118012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:01.118012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:01.118012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:01.118012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:01.118012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:01.118012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:01.118012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:01.118012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:01.118012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.118012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:01.118012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:01.118012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.118012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:01.118368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.118368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:01.118576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:01.119023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:01.119560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:01.119560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:01.119560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:05.588256 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:05.589006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:05.589006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:05.590167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:05.621644 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:05.622092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:05.623159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:05.623159 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:05.623159 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:05.623159 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:05.623159 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:05.623159 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:05.623159 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:05.623159 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:05.624229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:05.624229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:05.624229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:05.624229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:05.624229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:05.624229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:05.624229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:05.624229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:05.624229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:05.624229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:05.624229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:05.624229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:05.624774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:05.624774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:05.624774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:05.624774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:05.624774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:05.624774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:05.624774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:05.624774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:05.624774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:05.624774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:05.624774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:05.624774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:05.624774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:05.625356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:05.626008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:05.626795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:05.626795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:05.626795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:05.626795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:05.626795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:05.626795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:06.048951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:06.049301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:06.049301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:06.049982 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:06.050244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.050896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:06.050896 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.050896 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:06.050896 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:06.050896 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.050896 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:06.050896 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.050896 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:06.052557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.052557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:06.052557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:06.052557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:06.052557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:06.052557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:06.052557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:06.052557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.052557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:06.052557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:06.052557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:06.052557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.054244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:06.054244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:06.054244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:06.054244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:06.054244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:06.054244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:06.054244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:06.054244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:06.054244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.054244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:06.054244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:06.054244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.054244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:06.055509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:06.056179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.056726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:06.056726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:06.056726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.056726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:06.056726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.056726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:06.057611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:06.057837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:06.057945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:06.058068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:27:06.058604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:06.058830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.059401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:06.059401 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.059401 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:06.059401 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:06.059401 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.059401 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:06.059401 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.059401 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:06.060121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:06.060560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:06.060560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:06.060560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.060560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:06.060560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:06.060560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.060560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:06.060560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.060560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:06.061061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.061061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:06.061061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:06.061061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:06.061061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:06.061061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:06.061061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:06.061061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:27:06.061061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:06.061061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:06.061061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.061061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:06.061779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:06.062567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:06.063054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:06.063639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:06.063639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:06.063639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:06.063639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:06.063639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:06.063639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.063639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:06.063639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:06.063639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.063639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:06.063639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.063639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:06.106058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:06.106483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:06.106688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:06.106880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:06.107447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:06.107736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.108413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:06.108413 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.108413 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:06.108413 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:06.108413 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.108413 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:06.108413 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.108413 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:06.109607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:06.110277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:06.110277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:06.110277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.110277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:06.110277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:06.110277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.110277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:06.110277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.110277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:06.110947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.110947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:06.110947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:06.110947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:06.110947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:06.110947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:06.110947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:06.110947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:06.110947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:06.110947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.110947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:06.111464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:06.112079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.112079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:06.112079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.112079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:06.112659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:06.113420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:06.114222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.114222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:06.114222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.114222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:06.115516 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:06.115882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:06.116047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:06.116322 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:06.116831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:06.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:06.117232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.117805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:06.117805 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.117805 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:06.117805 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:06.117805 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.117805 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:06.117805 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.117805 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:06.118627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.118627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:06.118627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:06.118627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:06.118627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:06.118627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:06.118627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:06.118627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.118627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:06.118627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:06.118627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:06.118627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.119019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:06.119019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:06.119019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:06.119019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:06.119019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:06.119019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:06.119019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:06.119019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:06.119019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.119019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:06.119019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:06.119019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.119019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:06.119445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.119445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:06.119545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:06.120010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:06.120623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:06.120623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:06.120623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:10.588894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:10.589878 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:10.589878 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:10.591303 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:10.628116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:10.628591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:10.629416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:10.629416 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:10.629416 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:10.629416 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:10.629416 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:10.629416 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:10.629416 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:10.629416 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:10.630368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:10.630368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:10.630368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:10.630368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:10.630368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:10.630368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:10.630368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:10.630368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:10.630368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:10.630368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:10.630368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:10.630368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:10.630970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:10.630970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:10.630970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:10.630970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:10.630970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:10.630970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:10.630970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:10.630970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:10.630970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:10.630970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:10.630970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:10.630970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:10.630970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:10.631648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:10.632409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:10.633313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:10.633313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:10.633313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:10.633313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:10.633313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:10.633313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:11.049679 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:11.050166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:11.050390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:11.051362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:11.051663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.052479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:11.052479 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.052479 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:11.052479 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:11.052479 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.052479 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:11.052479 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.052479 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:11.054192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.054192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:11.054192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:11.054192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:11.054192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:11.054192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:11.054192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:11.054192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.054192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:11.054192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:11.054192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:11.054192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:11.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:11.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:11.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:11.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:11.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:11.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:11.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:11.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:11.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:11.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:11.055478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.055478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:11.056037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:11.056738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.057603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:11.057603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.057603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:11.058021 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:11.058288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:11.058379 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:11.058497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:27:11.058920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:11.059237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.059896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:11.059896 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.059896 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:11.059896 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:11.059896 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.059896 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:11.059896 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.059896 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:11.060603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:11.061367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:11.061367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:11.061367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.061367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:11.061367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:11.061367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.061367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:11.061367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.061367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:11.061815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.061815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:11.061815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:11.061815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:11.061815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:11.061815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:11.061815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:11.061815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:27:11.061815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:11.061815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:11.061815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.061815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:11.062192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:11.062617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:11.063028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:11.063573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:11.063573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:11.063573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:11.063573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:11.063573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:11.063573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.063573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:11.063573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:11.063573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.063573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:11.063573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.063573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:11.106616 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:11.106997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:11.107220 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:11.107503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:11.108028 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:11.108415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.109134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:11.109134 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.109134 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:11.109134 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:11.109134 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.109134 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:11.109134 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.109134 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:11.110607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:11.111316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:11.111316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:11.111316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.111316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:11.111316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:11.111316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.111316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:11.111316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.111316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:11.112366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.112366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:11.112366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:11.112366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:11.112366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:11.112366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:11.112366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:11.112366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:11.112366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:11.112366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.112366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:11.112939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:11.113619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.113619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:11.113619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.113619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:11.114622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:11.115495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:11.116373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.116373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:11.116373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.116373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:11.116373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:11.116373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:11.116666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:11.116798 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:11.117236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:11.117572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:11.117736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.118409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:11.118409 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.118409 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:11.118409 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:11.118409 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.118409 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:11.118409 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.118409 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:11.119279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.119279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:11.119279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:11.119279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:11.119279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:11.119279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:11.119279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:11.119279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.119279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:11.119279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:11.119279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:11.119279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.119651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:11.119651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:11.119651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:11.119651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:11.119651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:11.119651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:11.119651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:11.119651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:11.119651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.119651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:11.119651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:11.119651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.119651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:11.120021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.120021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:11.120467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:11.120958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:11.121518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:11.121518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:11.121518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:15.589617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:15.590443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:15.590443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:15.591636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:15.621302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:15.621454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:15.621916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:15.621916 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:15.621916 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:15.621916 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:15.621916 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:15.621916 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:15.621916 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:15.621916 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:15.622604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:15.622604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:15.622604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:15.622604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:15.622604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:15.622604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:15.622604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:15.622604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:15.622604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:15.622604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:15.622604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:15.622604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:15.623169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:15.623169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:15.623169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:15.623169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:15.623169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:15.623169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:15.623169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:15.623169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:15.623169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:15.623169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:15.623169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:15.623169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:15.623169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:15.623730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:15.624447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:15.625193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:15.625193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:15.625193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:15.625193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:15.625193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:15.625193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:16.050414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:16.051137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:16.051137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:16.051827 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:16.052104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.053944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:16.053944 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.053944 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:16.053944 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:16.053944 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.053944 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:16.053944 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.053944 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:16.054729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.054729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:16.054729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:16.054729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:16.054729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:16.054729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:16.054729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:16.054729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.054729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:16.054729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:16.054729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:16.054729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.056175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:16.056175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:16.056175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:16.056175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:16.056175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:16.056175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:16.056175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:16.056175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:16.056175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.056175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:16.056175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:16.056175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.056175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:16.056577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:16.056988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.057521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:16.057521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:16.057521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.057521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:16.057521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.057521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:16.058438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:16.058752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:16.058859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:16.058958 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:27:16.059532 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:16.059823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.060612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:16.060612 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.060612 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:16.060612 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:16.060612 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.060612 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:16.060612 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.060612 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:16.061396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:16.062275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:16.062690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:16.062690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:16.062690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:27:16.062690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:16.062690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:16.062690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.062690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:16.062690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:16.062690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:16.062690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.062690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:16.062690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:16.062690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:16.063867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:16.063867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:16.063867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:16.063867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:16.063867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:16.063867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.063867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:16.063867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:16.063867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.063867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:16.063867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.063867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:16.064258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:16.064730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:16.065151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.065151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:16.065151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:16.065151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.065151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:16.065151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.065151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:16.107063 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:16.107754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:16.107951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:16.108027 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:16.108710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:16.109112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.109937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:16.109937 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.109937 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:16.109937 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:16.109937 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.109937 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:16.109937 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.109937 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:16.110854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:16.111675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:16.112457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:16.112457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:16.112457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:16.112457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:16.112457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.112457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:16.112457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:16.112457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:16.112457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.112457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:16.112457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:16.112457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:16.112457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:16.113056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:16.113056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:16.113056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:16.113056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:16.113056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.113056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:16.113056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:16.113056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.113056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:16.113056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.113056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:16.113056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:16.113602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.114635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:16.116446 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:16.116944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:16.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:16.117282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:16.117714 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:16.118068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:16.118283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.119293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:16.119293 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.119293 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:16.119293 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:16.119293 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.119293 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:16.119293 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.119293 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:16.120342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.120342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:16.120342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:16.120342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:16.120342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:16.120342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:16.120342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:16.120342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.120342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:16.120342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:16.120342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:16.120342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.120770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:16.120770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:16.120770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:16.120770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:16.120770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:16.120770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:16.120770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:16.120770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:16.120770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.120770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:16.120770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:16.120770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.120770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:16.121139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.121139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:16.121299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:16.121827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:16.122353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:16.122353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:16.122353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:20.590180 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:20.590931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:20.590931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:20.592068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:20.628376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:20.628376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:20.629925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:20.630545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:20.630935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:20.632654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:20.633371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:20.633371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:20.633371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:20.633371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:20.633371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:20.633371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:20.633371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:20.633371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:20.633371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:21.051009 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:21.051666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:21.051857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:21.052991 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:21.053949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.054927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:21.054927 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.054927 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:21.054927 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:21.054927 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.054927 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:21.054927 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.054927 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:21.056437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.056437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:21.056437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:21.056437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:21.056437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:21.056437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:21.056437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:21.056437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.056437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:21.056437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:21.056437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:21.056437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.057017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:21.057017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:21.057017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:21.057017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:21.057017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:21.057017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:21.057017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:21.057017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:21.057017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.057017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:21.057017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:21.057017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.057017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:21.057615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.057615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:21.058148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:21.058981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.059500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:21.059500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.059500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:21.059500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:21.059500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:21.059500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:21.059751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:27:21.060013 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:21.060276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.060822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:21.060822 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.060822 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:21.060822 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:21.060822 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.060822 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:21.060822 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.060822 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:21.061628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:21.062103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:21.062103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:21.062103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.062103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:21.062103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:21.062103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.062103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:21.062103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.062103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:21.062511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.062511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:21.062511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:21.062511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:21.062511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:21.062511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:21.062511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:21.062511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:27:21.062511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:21.062511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:21.062511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.062511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:21.062903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:21.063396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.063396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:21.063396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.063396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:21.063746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:21.064198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:21.064726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.064726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:21.064726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:21.064726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.064726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:21.064726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.064726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:21.107697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:21.108180 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:21.108394 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:21.108580 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:21.109127 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:21.109620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.110460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:21.110460 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.110460 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:21.110460 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:21.110460 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.110460 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:21.110460 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.110460 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:21.111778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:21.112506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:21.112506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:21.112506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.112506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:21.112506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:21.112506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.112506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:21.112506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.112506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:21.113139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.113139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:21.113139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:21.113139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:21.113139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:21.113139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:21.113139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:21.113139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:21.113139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:21.113139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.113139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:21.113810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:21.114461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.114461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:21.114461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.114461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:21.114904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:21.115697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:21.116505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.116505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:21.116505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.116505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:21.117593 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:21.117593 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:21.117593 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:21.117837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:21.118197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:21.118815 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:21.119046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.119694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:21.119694 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.119694 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:21.119694 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:21.119694 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.119694 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:21.119694 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.119694 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:21.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:21.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:21.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:21.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:21.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:21.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:21.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:21.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:21.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:21.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.121083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:21.121083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:21.121083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:21.121083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:21.121083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:21.121083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:21.121083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:21.121083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:21.121083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.121083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:21.121083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:21.121083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.121083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:21.121452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.121452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:21.123053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:21.123545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:21.124066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:21.124066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:21.124066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:25.590906 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:25.591650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:25.591718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:25.592932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:25.624376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:25.624671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:25.625407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:25.625407 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:25.625407 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:25.625407 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:25.625407 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:25.625407 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:25.625407 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:25.625407 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:25.626241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:25.626241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:25.626241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:25.626241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:25.626241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:25.626241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:25.626241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:25.626241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:25.626241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:25.626241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:25.626241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:25.626241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:25.626779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:25.626779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:25.626779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:25.626779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:25.626779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:25.626779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:25.626779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:25.626779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:25.626779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:25.626779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:25.626779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:25.626779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:25.626779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:25.627359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:25.628013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:25.628872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:25.628872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:25.628872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:25.628872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:25.628872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:25.628872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:26.051749 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:26.052600 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:26.052800 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:26.053838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:26.054242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.055103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:26.055103 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.055103 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:26.055103 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:26.055103 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.055103 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:26.055103 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.055103 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:26.056469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.056469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:26.056469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:26.056469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:26.056469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:26.056469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:26.056469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:26.056469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.056469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:26.056469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:26.056469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:26.056469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.057009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:26.057009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:26.057009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:26.057009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:26.057009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:26.057009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:26.057009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:26.057009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:26.057009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.057009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:26.057009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:26.057009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.057009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:26.057563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.057563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:26.057868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:26.058405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.058946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:26.058946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.058946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:26.059327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:26.060385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:26.060511 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:26.060656 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:27:26.061158 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:26.061821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.062826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:26.062826 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.062826 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:26.062826 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:26.062826 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.062826 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:26.062826 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.062826 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:26.063096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:26.063543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:26.063543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:26.063543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.063543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:26.063543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:26.063543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.063543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:26.063543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.063543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:26.063982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.063982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:26.063982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:26.063982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:26.063982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:26.063982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:26.063982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:26.063982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:27:26.063982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:26.063982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:26.063982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.063982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:26.064396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:26.064782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.064782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:26.064782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.064782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:26.065052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:26.065500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:26.066058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.066058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:26.066058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:26.066058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.066058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:26.066058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.066058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:26.108265 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:26.109059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:26.109263 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:26.109487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:26.110122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:26.110516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.111338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:26.111338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.111338 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:26.111338 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:26.111338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.111338 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:26.111338 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.111338 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:26.112862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:26.113535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:26.113535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:26.113535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.113535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:26.113535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:26.113535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.113535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:26.113535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.113535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:26.114326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.114326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:26.114326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:26.114326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:26.114326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:26.114326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:26.114326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:26.114326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:26.114326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:26.114326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.114326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:26.114832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:26.115626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.115626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:26.115626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.115626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:26.116037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:26.117083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:26.118404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.118404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:26.118404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.118404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:26.118404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:26.118404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:26.118404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:26.118404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:26.119558 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:26.119947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:26.120130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.121456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:26.121456 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.121456 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:26.121456 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:26.121456 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.121456 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:26.121456 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.121456 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:26.123447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.123447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:26.123447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:26.123447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:26.123447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:26.123447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:26.123447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:26.123447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.123447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:26.123447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:26.123447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:26.123447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.123784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:26.123784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:26.123784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:26.123784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:26.123784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:26.123784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:26.123784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:26.123784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:26.123784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.123784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:26.123784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:26.123784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.123784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:26.124119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.124119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:26.125242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:26.125689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:26.126193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:26.126193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:26.126193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:30.591579 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:30.592466 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:30.592671 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:30.594069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:30.619712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:30.620115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:30.621008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:30.621008 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:30.621008 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:30.621008 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:30.621008 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:30.621008 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:30.621008 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:30.621008 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:30.621794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:30.621794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:30.621794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:30.621794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:30.621794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:30.621794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:30.621794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:30.621794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:30.621794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:30.621794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:30.621794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:30.621794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:30.622406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:30.622406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:30.622406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:30.622406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:30.622406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:30.622406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:30.622406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:30.622406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:30.622406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:30.622406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:30.622406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:30.622406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:30.622406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:30.622944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:30.623588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:30.624382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:30.624382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:30.624382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:30.624382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:30.624382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:30.624382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:31.052857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:31.053662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:31.053867 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:31.054740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:31.055592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.056523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:31.056523 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.056523 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:31.056523 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:31.056523 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.056523 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:31.056523 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.056523 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:31.058142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.058142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:31.058142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:31.058142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:31.058142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:31.058142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:31.058142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:31.058142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.058142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:31.058142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:31.058142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:31.058142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.058479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:31.058479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:31.058479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:31.058479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:31.058479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:31.058479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:31.058479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:31.058479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:31.058479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.058479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:31.058479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:31.058479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.058479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:31.058744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.058744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:31.059920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:31.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.061281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:31.061281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.061281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:31.061281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:31.061281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:31.061281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:31.061281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:27:31.061598 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:31.062156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.062962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:31.062962 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.062962 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:31.062962 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:31.062962 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.062962 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:31.062962 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.062962 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:31.064238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:31.064671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:31.064671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:31.064671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.064671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:31.064671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:31.064671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.064671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:31.064671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.064671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:31.065601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.065601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:31.065601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:31.065601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:31.065601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:31.065601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:31.065601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:31.065601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:27:31.065601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:31.065601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:31.065601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.065601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:31.066030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:31.066639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.066639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:31.066639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.066639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:31.067236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:31.067931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:31.068547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.068547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:31.068547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:31.068547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.068547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:31.068547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.068547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:31.108945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:31.109426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:31.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:31.109833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:31.110442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:31.110895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.111892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:31.111892 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.111892 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:31.111892 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:31.111892 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.111892 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:31.111892 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.111892 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:31.113426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:31.114273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:31.114273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:31.114273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.114273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:31.114273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:31.114273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.114273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:31.114273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.114273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:31.115038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.115038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:31.115038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:31.115038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:31.115038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:31.115038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:31.115038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:31.115038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:31.115038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:31.115038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.115038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:31.115469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:31.115849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.115849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:31.115849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.115849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:31.117429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:31.117985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:31.118651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.118651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:31.118651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.118651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:31.118651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:31.118651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:31.118651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:31.118651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:31.119129 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:31.119537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:31.119734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:31.120578 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.120578 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:31.120578 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:31.120578 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.120578 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:31.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:31.121429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.121429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:31.121429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:31.121429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:31.121429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:31.121429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:31.121429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:31.121429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.121429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:31.121429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:31.121429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:31.121429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:31.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:31.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:31.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:31.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:31.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:31.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:31.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:31.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:31.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:31.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:31.122131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.122131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:31.122394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:31.122854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:31.123492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:31.123492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:31.123492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:35.592381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:35.593312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:35.593511 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:35.595021 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:35.624772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=259 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:27:35.628155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:35.628680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:35.630099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:35.630099 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:35.630099 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:35.630099 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:35.630099 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:35.630099 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:35.630099 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:35.630099 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:35.631521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:35.631521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:35.631521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:35.631521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:35.631521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:35.631521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:35.631521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:35.631521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:35.631521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:35.631521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:35.631521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:35.631521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:35.632434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:35.632434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:35.632434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:35.632434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:35.632434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:35.632434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:35.632434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:35.632434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:35.632434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:35.632434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:35.632434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:35.632434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:35.632434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:35.633189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:35.634144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:35.635360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:35.635360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:35.635360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:35.635360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:35.635360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:35.635360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:36.053662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:36.053993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:36.054107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:36.055047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:36.055507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.056325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:36.056325 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.056325 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:36.056325 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:36.056325 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.056325 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:36.056325 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.056325 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:36.057360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.057360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:36.057360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:36.057360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:36.057360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:36.057360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:36.057360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:36.057360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.057360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:36.057360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:36.057360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:36.057360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.057924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:36.057924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:36.057924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:36.057924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:36.057924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:36.057924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:36.057924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:36.057924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:36.057924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.057924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:36.057924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:36.057924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.057924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:36.058507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:36.059148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:36.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:36.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:36.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.060067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:36.060639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:36.060891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:36.060997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:36.061165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:27:36.061596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:36.061912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.062611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:36.062611 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.062611 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:36.062611 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:36.062611 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.062611 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:36.062611 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.062611 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:36.063372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:36.063854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:36.063854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:36.063854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.063854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:36.063854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:36.063854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.063854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:36.063854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.063854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:36.064196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.064196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:36.064196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:36.064196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:36.064196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:36.064196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:36.064196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:36.064196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:27:36.064196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:36.064196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:36.064196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.064196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:36.064608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:36.065494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:36.065896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:36.066794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:36.066794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:36.066794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:36.066794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:36.066794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:36.066794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.066794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:36.066794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:36.066794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.066794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:36.066794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.066794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:36.109532 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:36.109923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:36.110107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:36.110350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:36.110969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:36.111350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.112128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:36.112128 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.112128 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:36.112128 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:36.112128 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.112128 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:36.112128 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.112128 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:36.113319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:36.113982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:36.113982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:36.113982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.113982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:36.113982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:36.113982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.113982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:36.113982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.113982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:36.114795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.114795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:36.114795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:36.114795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:36.114795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:36.114795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:36.114795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:36.114795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:36.114795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:36.114795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.114795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:36.115331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:36.115936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.115936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:36.115936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.115936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:36.116124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:36.116925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:36.117750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.117750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:36.117750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.117750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:36.118671 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:36.118979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:36.119073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:36.119237 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:36.119699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:36.120070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:36.120312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:36.121030 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.121030 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:36.121030 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:36.121030 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.121030 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:36.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:36.121840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.121840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:36.121840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:36.121840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:36.121840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:36.121840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:36.121840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:36.121840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.121840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:36.121840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:36.121840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:36.121840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.122178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:36.122178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:36.122178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:36.122178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:36.122178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:36.122178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:36.122178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:36.122178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:36.122178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.122178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:36.122178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:36.122178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.122178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:36.122539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.122539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:36.122785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:36.123264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:36.123786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:36.123786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:36.123786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:40.593627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:40.594502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:40.594703 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:40.596110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:40.632496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:40.632828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:40.633747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:40.633747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:40.633747 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:40.633747 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:40.633747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:40.633747 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:40.633747 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:40.633747 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:40.634645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:40.634645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:40.634645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:40.634645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:40.634645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:40.634645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:40.634645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:40.634645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:40.634645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:40.634645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:40.634645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:40.634645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:40.635293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:40.635293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:40.635293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:40.635293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:40.635293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:40.635293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:40.635293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:40.635293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:40.635293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:40.635293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:40.635293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:40.635293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:40.635293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:40.635875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:40.636646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:40.637521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:40.637521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:40.637521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:40.637521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:40.637521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:40.637521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:41.054345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:41.054757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:41.054874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:41.057674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:41.057915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.058571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:41.058571 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.058571 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:41.058571 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:41.058571 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.058571 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:41.058571 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.058571 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:41.060406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.060406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:41.060406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:41.060406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:41.060406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:41.060406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:41.060406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:41.060406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.060406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:41.060406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:41.060406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:41.060406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.061045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:41.061045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:41.061045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:41.061045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:41.061045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:41.061045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:41.061045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:41.061045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:41.061045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.061045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:41.061045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:41.061045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.061045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:41.061657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:41.062381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.063274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:41.063274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:41.063274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.063274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:41.063274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.063274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:41.063274 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:41.063274 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:41.063274 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:41.063274 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:41.063785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.064462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:41.065365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.065365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:41.065365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.065365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:41.065365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:41.065365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:41.065365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:41.065365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:41.065365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:41.065365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:27:41.065365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:41.065365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:41.065756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:41.066133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.066508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.066983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:41.110383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:41.111172 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:41.111386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:41.111596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:41.113758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:41.114153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.115085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:41.115085 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.115085 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:41.115085 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:41.115085 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.115085 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:41.115085 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.115085 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:41.116471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:41.116962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:41.116962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:41.116962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.116962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:41.116962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:41.116962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.116962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:41.116962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.116962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:41.118191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.118191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:41.118191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:41.118191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:41.118191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:41.118191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:41.118191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:41.118191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:41.118191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:41.118191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.118191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:41.118521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:41.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:41.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:41.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:41.120678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:41.121496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.121496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:41.121496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.121496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:41.121496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:41.121496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:41.121496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:41.121496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:41.121877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:41.121877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:41.121982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.122554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:41.122554 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.122554 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:41.122554 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:41.122554 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.122554 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:41.122554 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.122554 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:41.123378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.123378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:41.123378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:41.123378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:41.123378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:41.123378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:41.123378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:41.123378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.123378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:41.123378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:41.123378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:41.123378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.124003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:41.124003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:41.124003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:41.124003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:41.124003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:41.124003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:41.124003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:41.124003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:41.124003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.124003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:41.124003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:41.124003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.124003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:41.124772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:41.125813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:41.127110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:41.127110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:41.127110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:41.127110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:41.127110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:41.127110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:45.594476 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:45.594953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:45.595156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:45.596513 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:45.628542 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:45.628987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:45.629993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:45.629993 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:45.629993 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:45.629993 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:45.629993 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:45.629993 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:45.629993 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:45.629993 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:45.631695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:45.631695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:45.631695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:45.631695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:45.631695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:45.631695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:45.631695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:45.631695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:45.631695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:45.631695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:45.631695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:45.631695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:45.632845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:45.632845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:45.632845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:45.632845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:45.632845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:45.632845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:45.632845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:45.632845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:45.632845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:45.632845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:45.632845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:45.632845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:45.632845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:45.633785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:45.634972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:45.635852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:45.635852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:45.635852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:45.635852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:45.635852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:45.635852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:46.054852 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:46.055379 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:46.055680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:46.056724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:46.057079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.058001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:46.058001 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.058001 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:46.058001 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:46.058001 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.058001 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:46.058001 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.058001 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:46.059461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.059461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:46.059461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:46.059461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:46.059461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:46.059461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:46.059461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:46.059461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.059461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:46.059461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:46.059461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:46.059461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.060040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:46.060040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:46.060040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:46.060040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:46.060040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:46.060040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:46.060040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:46.060040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:46.060040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.060040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:46.060040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:46.060040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.060040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:46.060640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.060640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:46.061230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:46.061927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.062765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:46.062765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.062765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:46.063310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:46.063604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:46.063763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:46.063978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:27:46.064414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:46.064658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.065339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:46.065339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.065339 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:46.065339 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:46.065339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.065339 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:46.065339 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.065339 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:46.066276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:46.066669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:46.066669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:46.066669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.066669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:46.066669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:46.066669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.066669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:46.066669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.066669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:46.067124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.067124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:46.067124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:46.067124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:46.067124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:46.067124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:46.067124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:46.067124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:27:46.067124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:46.067124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:46.067124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.067124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:46.067469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:46.067841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.067841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:46.067841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.067841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:46.068473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:46.069134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:46.069714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.069714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:46.069714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:46.069714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.069714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:46.069714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.069714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:46.111201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:46.111659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:46.111828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:46.112053 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:46.112983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:46.113152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.114251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:46.114251 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.114251 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:46.114251 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:46.114251 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.114251 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:46.114251 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.114251 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:46.117252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:46.117964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:46.118624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:46.118624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:46.118624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:46.118624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:46.118624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.118624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:46.118624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:46.118624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:46.118624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.118624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:46.118624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:46.118624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:46.118624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:46.119170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:46.119170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:46.119170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:46.119170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:46.119170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.119170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:46.119170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:46.119170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.119170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:46.119170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.119170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:46.119170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:46.119738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.120610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:46.123347 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:46.123809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:46.123918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:46.124074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:46.124707 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:46.124933 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:46.125054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.125648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:46.125648 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.125648 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:46.125648 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:46.125648 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.125648 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:46.125648 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.125648 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:46.126914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.126914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:46.126914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:46.126914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:46.126914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:46.126914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:46.126914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:46.126914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.126914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:46.126914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:46.126914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:46.126914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.127169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:46.127169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:46.127169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:46.127169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:46.127169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:46.127169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:46.127169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:46.127169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:46.127169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.127169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:46.127169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:46.127169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.127169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:46.127418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.127418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:46.128448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:46.128782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:46.129126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:46.129126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:46.129126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:50.595404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:50.595913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:50.595913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:50.597452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:50.623748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:50.624147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:50.625058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:50.625058 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:50.625058 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:50.625058 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:50.625058 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:50.625058 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:50.625058 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:50.625058 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:50.625984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:50.625984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:50.625984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:50.625984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:50.625984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:50.625984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:50.625984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:50.625984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:50.625984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:50.625984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:50.625984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:50.625984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:50.626576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:50.626576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:50.626576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:50.626576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:50.626576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:50.626576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:50.626576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:50.626576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:50.626576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:50.626576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:50.626576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:50.626576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:50.626576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:50.627075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:50.627735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:50.628594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:50.628594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:50.628594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:50.628594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:50.628594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:50.628594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:51.055729 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:51.056272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:51.056487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:51.057703 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:51.057945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.058596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:51.058596 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.058596 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:51.058596 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:51.058596 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.058596 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:51.058596 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.058596 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:51.059769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.059769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:51.059769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:51.059769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:51.059769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:51.059769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:51.059769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:51.059769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.059769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:51.059769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:51.059769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:51.059769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.060429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:51.060429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:51.060429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:51.060429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:51.060429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:51.060429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:51.060429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:51.060429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:51.060429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.060429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:51.060429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:51.060429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.060429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:51.061020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:51.061781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.062687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:51.062687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:51.062687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.062687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:51.062687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.062687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:51.063593 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:51.063884 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:51.064017 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:51.064139 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:27:51.064497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:51.064707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.065171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:51.065171 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.065171 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:51.065171 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:51.065171 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.065171 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:51.065171 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.065171 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:51.065992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:51.066443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:51.066443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:51.066443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.066443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:51.066443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:51.066443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.066443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:51.066443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.066443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:51.066849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.066849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:51.066849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:51.066849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:51.066849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:51.066849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:51.066849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:51.066849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:27:51.066849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:51.066849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:51.066849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.066849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:51.067339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:51.068153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:51.068646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:51.069150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:51.069150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:51.069150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:51.069150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:51.069150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:51.069150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.069150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:51.069150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:51.069150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.069150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:51.069150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.069150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:51.111537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:51.112116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:51.112351 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:51.112549 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:51.113036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:51.113524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.114166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:51.114166 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.114166 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:51.114166 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:51.114166 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.114166 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:51.114166 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.114166 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:51.115350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:51.116080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:51.116080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:51.116080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.116080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:51.116080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:51.116080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.116080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:51.116080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.116080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:51.117336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.117336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:51.117336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:51.117336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:51.117336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:51.117336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:51.117336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:51.117336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:51.117336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:51.117336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.117336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:51.117853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:51.118486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.118486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:51.118486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.118486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:51.120016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:51.120843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.120843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:51.120843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.120843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:51.123896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:51.124176 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:51.124296 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:51.124457 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:51.124703 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:51.124956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:51.125144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.125830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:51.125830 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.125830 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:51.125830 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:51.125830 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.125830 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:51.125830 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.125830 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:51.126653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.126653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:51.126653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:51.126653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:51.126653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:51.126653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:51.126653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:51.126653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.126653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:51.126653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:51.126653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:51.126653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:51.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:51.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:51.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:51.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:51.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:51.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:51.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:51.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:51.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:51.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:51.127476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.127476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:51.127657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:51.128102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:51.128639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:51.128639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:51.128639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:55.595869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:55.596761 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:55.596761 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:55.597957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:55.631474 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:55.631865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:55.632766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:55.632766 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:55.632766 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:55.632766 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:55.632766 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:55.632766 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:55.632766 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:55.632766 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:55.633510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:55.633510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:55.633510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:55.633510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:55.633510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:55.633510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:55.633510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:55.633510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:55.633510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:55.633510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:55.633510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:55.633510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:55.634079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:55.634079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:55.634079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:55.634079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:55.634079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:55.634079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:55.634079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:55.634079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:55.634079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:55.634079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:55.634079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:55.634079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:55.634079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:55.634657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:55.635323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:55.636072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:55.636072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:55.636072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:55.636072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:55.636072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:55.636072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:56.056459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:56.056844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:56.057032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:56.057920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:56.058191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.059024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:56.059024 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.059024 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:56.059024 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:56.059024 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.059024 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:56.059024 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.059024 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:56.060510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.060510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:56.060510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:56.060510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:56.060510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:56.060510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:56.060510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:56.060510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.060510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:56.060510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:56.060510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:56.060510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.061111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:56.061111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:56.061111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:56.061111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:56.061111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:56.061111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:56.061111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:56.061111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:56.061111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.061111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:56.061111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:56.061111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.061111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:56.061734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.061734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:56.062165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:56.063198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.064019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:56.064019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.064019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:56.064019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:56.064329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:56.064473 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:56.064590 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:27:56.064982 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:56.065355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.065954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:56.065954 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.065954 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:56.065954 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:56.065954 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.065954 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:56.065954 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.065954 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:56.066732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:56.067139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:56.067139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:56.067139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.067139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:56.067139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:56.067139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.067139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:56.067139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.067139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:56.067551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.067551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:56.067551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:56.067551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:56.067551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:56.067551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:56.067551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:56.067551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:27:56.067551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:56.067551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:56.067551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.067551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:56.067873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:56.068257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.068257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:56.068257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.068257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:56.068607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:56.069050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:56.069529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.069529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:56.069529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:56.069529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.069529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:56.069529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.069529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:56.112129 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:56.112459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:56.112549 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:56.112701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:56.113093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:56.113376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.113980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:56.113980 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.113980 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:56.113980 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:56.113980 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.113980 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:56.113980 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.113980 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:27:56.115116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:27:56.115806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:56.115806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:27:56.115806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.115806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:27:56.115806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:27:56.115806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.115806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:27:56.115806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.115806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:27:56.116519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.116519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:56.116519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:56.116519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:56.116519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:56.116519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:56.116519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:56.116519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:56.116519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:56.116519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.116519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:56.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:56.117697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.117697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:56.117697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.117697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:56.118343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:56.119074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:56.119865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.119865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:56.119865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.119865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:27:56.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:27:56.124647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:27:56.124820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:27:56.125033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:27:56.125447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:56.125786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:27:56.125968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.126610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:27:56.126610 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.126610 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:27:56.126610 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:27:56.126610 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.126610 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:27:56.126610 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.126610 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:27:56.127389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.127389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:27:56.127389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:56.127389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:27:56.127389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:27:56.127389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:56.127389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:56.127389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.127389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:56.127389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:56.127389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:27:56.127389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.127725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:27:56.127725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:56.127725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:27:56.127725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:56.127725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:27:56.127725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:27:56.127725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:56.127725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:27:56.127725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.127725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:27:56.127725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:27:56.127725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.127725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:27:56.128085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.128085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:27:56.128321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:27:56.128753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:27:56.129375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:27:56.129375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:27:56.129375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:00.596416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:00.597107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:00.597167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:00.598392 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:00.634913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:00.635195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:00.636092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:00.636092 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:00.636092 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:00.636092 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:00.636092 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:00.636092 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:00.636092 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:00.636092 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:00.637931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:00.637931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:00.637931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:00.637931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:00.637931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:00.637931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:00.637931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:00.637931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:00.637931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:00.637931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:00.637931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:00.637931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:00.638595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:00.638595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:00.638595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:00.638595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:00.638595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:00.638595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:00.638595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:00.638595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:00.638595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:00.638595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:00.638595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:00.638595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:00.638595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:00.639372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:00.639372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:00.639793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:00.640966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:00.642272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:00.642272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:00.642272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:01.057059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:01.057913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:01.058109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:01.059103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:01.059773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.060714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:01.060714 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.060714 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:01.060714 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:01.060714 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.060714 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:01.060714 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.060714 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:01.061767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.061767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:01.061767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:01.061767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:01.061767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:01.061767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:01.061767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:01.061767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.061767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:01.061767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:01.061767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:01.061767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.062119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:01.062119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:01.062119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:01.062119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:01.062119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:01.062119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:01.062119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:01.062119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:01.062119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.062119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:01.062119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:01.062119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.062119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:01.062518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.062518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:01.062600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:01.063035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.063633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:01.063633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.063633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:01.064646 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:01.064999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:01.065184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:01.065449 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:28:01.066463 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:01.066853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.068029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:01.068029 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.068029 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:01.068029 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:01.068029 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.068029 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:01.068029 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.068029 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:01.069771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:01.070911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:01.070911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:01.070911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.070911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:01.070911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:01.070911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.070911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:01.070911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.070911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:01.071378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.071378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:01.071378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:01.071378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:01.071378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:01.071378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:01.071378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:01.071378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:28:01.071378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:01.071378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:01.071378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.071378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:01.071992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:01.072685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.072685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:01.072685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.072685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:01.073058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:01.073819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:01.074644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.074644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:01.074644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:01.074644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.074644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:01.074644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.074644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:01.112607 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:01.113180 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:01.113425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:01.113600 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:01.114272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:01.114752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.115452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:01.115452 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.115452 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:01.115452 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:01.115452 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.115452 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:01.115452 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.115452 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:01.116681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:01.117084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:01.117084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:01.117084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.117084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:01.117084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:01.117084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.117084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:01.117084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.117084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:01.117939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.117939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:01.117939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:01.117939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:01.117939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:01.117939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:01.117939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:01.117939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:01.117939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:01.117939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.117939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:01.118294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:01.118798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.118798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:01.118798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.118798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:01.119640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:01.120327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:01.121333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.121333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:01.121333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.121333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:01.124695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:01.124861 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:01.124966 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:01.125114 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:01.125565 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:01.125939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:01.126137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.126892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:01.126892 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.126892 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:01.126892 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:01.126892 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.126892 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:01.126892 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.126892 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:01.127737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.127737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:01.127737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:01.127737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:01.127737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:01.127737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:01.127737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:01.127737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.127737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:01.127737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:01.127737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:01.127737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.128128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:01.128128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:01.128128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:01.128128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:01.128128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:01.128128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:01.128128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:01.128128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:01.128128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.128128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:01.128128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:01.128128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.128128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:01.128635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.128635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:01.128949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:01.129445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:01.129971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:01.129971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:01.129971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:05.597140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:05.598144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:05.598144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:05.599070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:05.633920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:05.634363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:05.635179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:05.635179 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:05.635179 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:05.635179 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:05.635179 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:05.635179 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:05.635179 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:05.635179 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:05.635957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:05.635957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:05.635957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:05.635957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:05.635957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:05.635957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:05.635957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:05.635957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:05.635957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:05.635957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:05.635957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:05.635957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:05.636536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:05.636536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:05.636536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:05.636536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:05.636536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:05.636536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:05.636536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:05.636536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:05.636536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:05.636536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:05.636536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:05.636536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:05.636536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:05.637290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:05.637990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:05.638846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:05.638846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:05.638846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:05.638846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:05.638846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:05.638846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:06.057987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:06.058921 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:06.059105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:06.060136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:06.060490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.061394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:06.061394 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.061394 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:06.061394 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:06.061394 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.061394 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:06.061394 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.061394 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:06.063009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.063009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:06.063009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:06.063009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:06.063009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:06.063009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:06.063009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:06.063009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.063009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:06.063009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:06.063009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:06.063009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:06.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:06.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:06.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:06.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:06.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:06.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:06.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:06.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:06.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:06.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:06.064317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.064317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:06.064656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:06.065162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.065746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:06.065746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.065746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:06.065746 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:06.065954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:06.066091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:06.066215 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:28:06.066694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:06.067069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.067902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:06.067902 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.067902 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:06.067902 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:06.067902 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.067902 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:06.067902 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.067902 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:06.068817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:06.069398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:06.069398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:06.069398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.069398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:06.069398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:06.069398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.069398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:06.069398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.069398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:06.069755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.069755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:06.069755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:06.069755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:06.069755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:06.069755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:06.069755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:06.069755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:28:06.069755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:06.069755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:06.069755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.069755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:06.070138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:06.070593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.070593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:06.070593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.070593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:06.070908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:06.071532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:06.072115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.072115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:06.072115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:06.072115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.072115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:06.072115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.072115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:06.113787 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:06.114352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:06.114524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:06.114719 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:06.115298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:06.115767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.116530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:06.116530 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.116530 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:06.116530 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:06.116530 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.116530 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:06.116530 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.116530 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:06.118305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:06.118305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:06.118305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.118305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:06.118305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:06.118305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.118305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:06.118305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.118305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:06.119102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.119102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:06.119102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:06.119102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:06.119102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:06.119102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:06.119102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:06.119102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:06.119102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:06.119102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.119102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:06.119470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:06.119886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.119886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:06.119886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.119886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:06.120755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:06.121280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:06.121948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.121948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:06.121948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.121948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:06.125116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:06.125391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:06.125503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:06.125657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:06.126140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:06.126533 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:06.126723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.127391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:06.127391 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.127391 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:06.127391 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:06.127391 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.127391 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:06.127391 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.127391 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:06.128222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.128222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:06.128222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:06.128222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:06.128222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:06.128222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:06.128222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:06.128222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.128222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:06.128222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:06.128222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:06.128222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.128666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:06.128666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:06.128666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:06.128666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:06.128666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:06.128666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:06.128666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:06.128666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:06.128666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.128666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:06.128666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:06.128666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.128666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:06.129117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.129117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:06.129189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:06.129733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:06.130399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:06.130399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:06.130399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:10.598004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:10.599290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:10.599290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:10.599906 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:10.633907 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:10.634339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:10.635237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:10.635237 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:10.635237 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:10.635237 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:10.635237 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:10.635237 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:10.635237 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:10.635237 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:10.636304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:10.636304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:10.636304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:10.636304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:10.636304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:10.636304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:10.636304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:10.636304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:10.636304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:10.636304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:10.636304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:10.636304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:10.636988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:10.636988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:10.636988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:10.636988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:10.636988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:10.636988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:10.636988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:10.636988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:10.636988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:10.636988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:10.636988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:10.636988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:10.636988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:10.637563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:10.638632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:10.639694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:10.639694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:10.639694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:10.639694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:10.639694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:10.639694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:11.059470 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:11.059947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:11.060104 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:11.061093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:11.061440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.062220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:11.062220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.062220 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:11.062220 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:11.062220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.062220 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:11.062220 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.062220 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:11.063786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.063786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:11.063786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:11.063786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:11.063786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:11.063786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:11.063786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:11.063786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.063786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:11.063786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:11.063786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:11.063786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.064379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:11.064379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:11.064379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:11.064379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:11.064379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:11.064379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:11.064379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:11.064379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:11.064379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.064379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:11.064379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:11.064379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.064379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:11.064996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.064996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:11.065416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:11.066437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.067182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:11.067182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.067182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:11.067182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:11.067182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:11.067182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:11.067182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:28:11.067505 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:11.068088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.068397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:11.068397 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.068397 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:11.068397 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:11.068397 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.068397 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:11.068397 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.068397 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:11.069520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:11.070396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:11.070396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:11.070396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.070396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:11.070396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:11.070396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.070396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:11.070396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.070396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:11.070900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.070900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:11.070900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:11.070900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:11.070900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:11.070900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:11.070900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:11.070900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:28:11.070900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:11.070900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:11.070900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.070900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:11.071566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:11.072013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.072013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:11.072013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.072013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:11.072121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:11.072544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:11.073002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.073002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:11.073002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:11.073002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.073002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:11.073002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.073002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:11.114613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:11.115010 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:11.115182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:11.115456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:11.116153 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:11.116548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.117246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:11.117246 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.117246 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:11.117246 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:11.117246 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.117246 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:11.117246 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.117246 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:11.118442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:11.119753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:11.119753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:11.119753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.119753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:11.119753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:11.119753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.119753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:11.119753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.119753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:11.120287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.120287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:11.120287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:11.120287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:11.120287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:11.120287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:11.120287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:11.120287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:11.120287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:11.120287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.120287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:11.120723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:11.121281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.121281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:11.121281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.121281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:11.121681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:11.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:11.124298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.124298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:11.124298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.124298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:11.125584 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:11.125865 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:11.125972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:11.126134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:11.126554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:11.126872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:11.127043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.127707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:11.127707 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.127707 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:11.127707 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:11.127707 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.127707 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:11.127707 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.127707 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:11.128591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.128591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:11.128591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:11.128591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:11.128591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:11.128591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:11.128591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:11.128591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.128591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:11.128591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:11.128591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:11.128591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.128907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:11.128907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:11.128907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:11.128907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:11.128907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:11.128907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:11.128907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:11.128907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:11.128907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.128907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:11.128907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:11.128907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.128907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:11.129254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.129254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:11.129634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:11.130512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:11.131443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:11.131443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:11.131443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:15.598770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:15.599928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:15.599928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:15.600552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:15.631102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:15.631424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:15.632022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:15.632022 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:15.632022 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:15.632022 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:15.632022 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:15.632022 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:15.632022 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:15.632022 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:15.632986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:15.632986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:15.632986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:15.632986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:15.632986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:15.632986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:15.632986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:15.632986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:15.632986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:15.632986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:15.632986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:15.632986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:15.633922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:15.633922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:15.633922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:15.633922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:15.633922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:15.633922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:15.633922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:15.633922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:15.633922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:15.633922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:15.633922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:15.633922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:15.633922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:15.634584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:15.635356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:15.635864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:15.635864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:15.635864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:15.635864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:15.635864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:15.635864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:16.060123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:16.060668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:16.060942 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:16.061766 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:16.062083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.062841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:16.062841 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.062841 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:16.062841 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:16.062841 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.062841 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:16.062841 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.062841 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:16.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:16.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:16.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:16.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:16.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:16.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:16.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:16.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:16.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:16.063769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.064746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:16.064746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:16.064746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:16.064746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:16.064746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:16.064746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:16.064746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:16.064746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:16.064746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.064746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:16.064746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:16.064746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.064746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:16.065546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:16.066394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.068165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:16.068165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:16.068165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.068165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:16.068165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.068165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:16.068165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:16.068165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:16.068165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:16.068165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:28:16.068964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:16.068964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.069820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:16.069820 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.069820 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:16.069820 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:16.069820 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.069820 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:16.069820 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.069820 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:16.070628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:16.071614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:16.072557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:16.072557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:16.072557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:28:16.072557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:16.072557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:16.072557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.072557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:16.072557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:16.072557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:16.072557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.072557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:16.072557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:16.072557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:16.073295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:16.073295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:16.073295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:16.073295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:16.073295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:16.073295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.073295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:16.073295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:16.073295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.073295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:16.073295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.073295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:16.073931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:16.074763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:16.075686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.075686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:16.075686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:16.075686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.075686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:16.075686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.075686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:16.115300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:16.115781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:16.115983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:16.116333 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:16.116987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:16.117615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.118394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:16.118394 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.118394 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:16.118394 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:16.118394 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.118394 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:16.118394 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.118394 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:16.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:16.120492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:16.120492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:16.120492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.120492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:16.120492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:16.120492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.120492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:16.120492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.120492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:16.121349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.121349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:16.121349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:16.121349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:16.121349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:16.121349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:16.121349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:16.121349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:16.121349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:16.121349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.121349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:16.122327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:16.122993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.122993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:16.122993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.122993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:16.123155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:16.123958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:16.125529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.125529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:16.125529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.125529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:16.126087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:16.126393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:16.126478 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:16.126666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:16.127096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:16.127496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:16.127685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.128361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:16.128361 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.128361 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:16.128361 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:16.128361 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.128361 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:16.128361 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.128361 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:16.129219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.129219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:16.129219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:16.129219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:16.129219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:16.129219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:16.129219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:16.129219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.129219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:16.129219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:16.129219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:16.129219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.129541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:16.129541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:16.129541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:16.129541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:16.129541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:16.129541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:16.129541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:16.129541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:16.129541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.129541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:16.129541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:16.129541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.129541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:16.129848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.129848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:16.130287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:16.131139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:16.131952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:16.131952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:16.131952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:20.599690 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:20.601108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:20.601108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:20.602146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:20.640341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:20.640694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:20.641589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:20.641589 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:20.641589 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:20.641589 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:20.641589 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:20.641589 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:20.641589 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:20.641589 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:20.642701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:20.642701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:20.642701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:20.642701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:20.642701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:20.642701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:20.642701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:20.642701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:20.642701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:20.642701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:20.642701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:20.642701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:20.643325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:20.643325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:20.643325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:20.643325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:20.643325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:20.643325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:20.643325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:20.643325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:20.643325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:20.643325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:20.643325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:20.643325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:20.643325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:20.643887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:20.644625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:20.645153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:20.645153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:20.645153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:20.645153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:20.645153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:20.645153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:21.060699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:21.060986 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:21.061165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:21.062124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:21.062353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.063745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:21.063745 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.063745 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:21.063745 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:21.063745 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.063745 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:21.063745 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.063745 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:21.065194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.065194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:21.065194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:21.065194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:21.065194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:21.065194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:21.065194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:21.065194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.065194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:21.065194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:21.065194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:21.065194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.065539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:21.065539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:21.065539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:21.065539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:21.065539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:21.065539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:21.065539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:21.065539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:21.065539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.065539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:21.065539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:21.065539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.065539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:21.065861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.065861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:21.066811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:21.067265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.067739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:21.067739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.067739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:21.067739 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:21.067849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:21.067899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:21.068049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:28:21.068495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:21.068731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.069325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:21.069325 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.069325 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:21.069325 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:21.069325 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.069325 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:21.069325 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.069325 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:21.070351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:21.070709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:21.070709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:21.070709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.070709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:21.070709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:21.070709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.070709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:21.070709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.070709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:21.071785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.071785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:21.071785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:21.071785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:21.071785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:21.071785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:21.071785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:21.071785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:28:21.071785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:21.071785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:21.071785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.071785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:21.072163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:21.072536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:21.072899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:21.073384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:21.073384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:21.073384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:21.073384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:21.073384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:21.073384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.073384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:21.073384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:21.073384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.073384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:21.073384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.073384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:21.116147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:21.116649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:21.116800 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:21.116965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:21.117737 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:21.118041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.118757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:21.118757 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.118757 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:21.118757 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:21.118757 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.118757 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:21.118757 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.118757 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:21.119963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:21.120376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:21.120376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:21.120376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.120376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:21.120376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:21.120376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.120376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:21.120376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.120376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:21.121168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.121168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:21.121168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:21.121168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:21.121168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:21.121168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:21.121168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:21.121168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:21.121168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:21.121168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.121168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:21.121469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:21.121808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.121808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:21.121808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.121808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:21.122764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:21.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:21.123633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.123633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:21.123633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.123633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:21.126396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:21.126554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:21.126663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:21.126798 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:21.127215 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:21.127528 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:21.127669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.128228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:21.128228 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.128228 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:21.128228 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:21.128228 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.128228 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:21.128228 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.128228 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:21.128939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.128939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:21.128939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:21.128939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:21.128939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:21.128939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:21.128939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:21.128939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.128939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:21.128939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:21.128939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:21.128939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.129227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:21.129227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:21.129227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:21.129227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:21.129227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:21.129227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:21.129227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:21.129227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:21.129227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.129227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:21.129227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:21.129227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.129227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:21.129515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.129515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:21.129917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:21.130314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:21.130752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:21.130752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:21.130752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:25.600742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:25.601700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:25.601700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:25.602743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:25.633760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:25.634241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:25.635256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:25.635256 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:25.635256 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:25.635256 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:25.635256 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:25.635256 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:25.635256 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:25.635256 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:25.636185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:25.636185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:25.636185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:25.636185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:25.636185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:25.636185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:25.636185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:25.636185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:25.636185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:25.636185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:25.636185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:25.636185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:25.636880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:25.636880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:25.636880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:25.636880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:25.636880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:25.636880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:25.636880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:25.636880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:25.636880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:25.636880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:25.636880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:25.636880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:25.636880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:25.637479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:25.638351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:25.639179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:25.639179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:25.639179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:25.639179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:25.639179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:25.639179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:26.061965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:26.061965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:26.061965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:26.062888 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:26.063387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.064033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:26.064033 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.064033 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:26.064033 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:26.064033 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.064033 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:26.064033 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.064033 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:26.065135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.065135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:26.065135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:26.065135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:26.065135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:26.065135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:26.065135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:26.065135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.065135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:26.065135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:26.065135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:26.065135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.065788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:26.065788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:26.065788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:26.065788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:26.065788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:26.065788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:26.065788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:26.065788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:26.065788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.065788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:26.065788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:26.065788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.065788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:26.066369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:26.067094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.068005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:26.068005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:26.068005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.068005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:26.068005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.068005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:26.068005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:26.068005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:26.068005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:26.068516 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:28:26.068516 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:26.068744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.069305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:26.069305 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.069305 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:26.069305 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:26.069305 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.069305 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:26.069305 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.069305 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:26.069864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:26.070370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:26.070370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:26.070370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.070370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:26.070370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:26.070370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.070370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:26.070370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.070370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:26.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:26.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:26.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:26.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:26.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:26.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:26.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:28:26.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:26.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:26.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:26.071018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:26.071422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:26.071843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:26.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:26.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:26.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:26.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:26.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:26.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:26.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:26.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:26.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:26.116578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:26.116765 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:26.116894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:26.117000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:26.118007 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:26.118594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.119750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:26.119750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.119750 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:26.119750 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:26.119750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.119750 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:26.119750 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.119750 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:26.120654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:26.121384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:26.121384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:26.121384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.121384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:26.121384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:26.121384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.121384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:26.121384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.121384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:26.121783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.121783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:26.121783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:26.121783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:26.121783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:26.121783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:26.121783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:26.121783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:26.121783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:26.121783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.121783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:26.122307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:26.122901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:26.123556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:26.124370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:26.124370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:26.124370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:26.124370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:26.124370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.124370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:26.124370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:26.124370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.124370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:26.124370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.124370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:26.126728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:26.126928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:26.126998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:26.127190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:26.127956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:26.128334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:26.128507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.129348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:26.129348 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.129348 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:26.129348 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:26.129348 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.129348 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:26.129348 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.129348 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:26.130190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.130190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:26.130190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:26.130190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:26.130190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:26.130190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:26.130190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:26.130190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.130190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:26.130190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:26.130190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:26.130190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.130600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:26.130600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:26.130600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:26.130600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:26.130600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:26.130600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:26.130600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:26.130600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:26.130600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.130600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:26.130600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:26.130600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.130600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:26.130964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.130964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:26.131161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:26.131660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:26.132227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:26.132227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:26.132227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:30.601592 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:30.602399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:30.602399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:30.603167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:30.639193 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:30.639637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:30.640922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:30.640922 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:30.640922 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:30.640922 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:30.640922 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:30.640922 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:30.640922 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:30.640922 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:30.642003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:30.642003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:30.642003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:30.642003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:30.642003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:30.642003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:30.642003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:30.642003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:30.642003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:30.642003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:30.642003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:30.642003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:30.643016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:30.643016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:30.643016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:30.643016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:30.643016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:30.643016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:30.643016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:30.643016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:30.643016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:30.643016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:30.643016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:30.643016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:30.643016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:30.643602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:30.644401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:30.645099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:30.645099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:30.645099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:30.645099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:30.645099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:30.645099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:31.062326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:31.062988 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:31.063182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:31.064264 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:31.064641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.065645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:31.065645 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.065645 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:31.065645 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:31.065645 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.065645 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:31.065645 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.065645 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:31.067186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.067186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:31.067186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:31.067186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:31.067186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:31.067186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:31.067186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:31.067186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.067186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:31.067186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:31.067186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:31.067186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.068323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:31.068323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:31.068323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:31.068323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:31.068323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:31.068323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:31.068323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:31.068323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:31.068323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.068323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:31.068323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:31.068323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.068323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:31.069486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:31.070087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.070849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:31.070849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:31.070849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.070849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:31.070849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.070849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:31.070849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:31.070849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:31.070849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:31.070849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:28:31.071359 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:31.071359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.071756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:31.071756 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.071756 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:31.071756 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:31.071756 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.071756 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:31.071756 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.071756 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:31.073036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:31.073489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:31.073489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:31.073489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.073489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:31.073489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:31.073489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.073489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:31.073489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.073489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:31.073866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.073866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:31.073866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:31.073866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:31.073866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:31.073866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:31.073866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:31.073866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:28:31.073866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:31.073866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:31.073866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.073866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:31.074201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:31.074571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.074571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:31.074571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.074571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:31.074860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:31.075310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:31.075773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.075773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:31.075773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:31.075773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.075773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:31.075773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.075773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:31.117173 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:31.117587 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:31.117737 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:31.117863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:31.118492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:31.118894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.119676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:31.119676 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.119676 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:31.119676 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:31.119676 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.119676 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:31.119676 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.119676 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:31.120461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:31.121158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:31.121158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:31.121158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.121158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:31.121158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:31.121158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.121158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:31.121158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.121158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:31.121565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.121565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:31.121565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:31.121565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:31.121565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:31.121565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:31.121565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:31.121565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:31.121565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:31.121565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.121565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:31.122074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:31.122702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.122702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:31.122702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.122702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:31.123013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:31.123924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:31.125088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.125088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:31.125088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.125088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:31.127167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:31.127642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:31.127822 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:31.128163 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:31.128606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:31.128822 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:31.129157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.129625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:31.129625 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.129625 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:31.129625 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:31.129625 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.129625 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:31.129625 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.129625 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:31.130451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.130451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:31.130451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:31.130451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:31.130451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:31.130451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:31.130451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:31.130451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.130451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:31.130451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:31.130451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:31.130451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.131065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:31.131065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:31.131065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:31.131065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:31.131065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:31.131065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:31.131065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:31.131065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:31.131065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.131065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:31.131065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:31.131065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.131065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:31.131696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:31.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:31.133316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:31.133316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:31.133316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:31.133316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:31.133316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:31.133316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:35.602437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:35.604199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:35.604427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:35.605873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:35.633438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=236 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:28:35.643698 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:35.644128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:35.647306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:35.647793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:35.648174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:35.648583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:35.649163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:35.649163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:35.649163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:35.649163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:35.649163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:35.649163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:35.649163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:35.649163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:35.649163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:36.063012 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:36.064146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:36.064146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:36.064757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:36.065066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:36.068042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:36.068820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:36.069377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:36.069705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:36.070162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:36.070162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:36.070162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:36.070162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.070162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:36.070162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:36.070162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.070162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:36.070162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.070162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:36.070162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:36.071936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:36.071936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:36.071936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:28:36.072438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:36.072722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.076235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:36.076757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.077145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:36.077145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:36.077145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:36.077145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:36.077145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:36.077145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:36.077145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:28:36.077145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:36.077145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:36.077145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.077145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:36.077145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:36.077145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.077477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:36.077801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.077801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:36.077882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:36.078285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:36.078715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.078715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:36.078715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:36.078715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.078715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:36.078715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.078715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:36.117767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:36.118368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:36.118557 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:36.118766 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:36.119344 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:36.119683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.120472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:36.120472 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.120472 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:36.120472 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:36.120472 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.120472 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:36.120472 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.120472 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:36.121755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:36.122151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:36.122151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:36.122151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.122151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:36.122151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:36.122151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.122151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:36.122151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.122151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:36.123130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.123130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:36.123130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:36.123130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:36.123130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:36.123130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:36.123130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:36.123130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:36.123130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:36.123130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.123130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:36.123450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:36.123792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.123792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:36.123792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.123792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:36.124920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:36.125417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:36.125861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.125861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:36.125861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.125861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:36.127520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:36.127710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:36.127821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:36.127959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:36.128432 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:36.128757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:36.128984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.129612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:36.129612 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.129612 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:36.129612 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:36.129612 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.129612 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:36.129612 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.129612 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:36.130429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.130429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:36.130429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:36.130429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:36.130429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:36.130429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:36.130429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:36.130429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.130429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:36.130429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:36.130429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:36.130429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.130986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:36.130986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:36.130986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:36.130986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:36.130986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:36.130986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:36.130986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:36.130986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:36.130986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.130986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:36.130986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:36.130986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.130986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:36.131593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:36.132281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:36.133155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:36.133155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:36.133155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:36.133155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:36.133155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:36.133155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:40.604654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:40.605796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:40.605796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:40.606328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:40.642952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:40.643388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:40.644282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:40.644282 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:40.644282 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:40.644282 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:40.644282 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:40.644282 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:40.644282 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:40.644282 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:40.645438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:40.645438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:40.645438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:40.645438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:40.645438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:40.645438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:40.645438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:40.645438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:40.645438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:40.645438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:40.645438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:40.645438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:40.646057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:40.646057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:40.646057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:40.646057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:40.646057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:40.646057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:40.646057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:40.646057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:40.646057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:40.646057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:40.646057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:40.646057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:40.646057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:40.646723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:40.647639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:40.648749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:40.648749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:40.648749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:40.648749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:40.648749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:40.648749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:41.064004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:41.065332 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:41.065332 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:41.065852 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:41.066104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.067000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:41.067000 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.067000 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:41.067000 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:41.067000 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.067000 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:41.067000 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.067000 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:41.069231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.069231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:41.069231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:41.069231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:41.069231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:41.069231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:41.069231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:41.069231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.069231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:41.069231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:41.069231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:41.069231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.069808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:41.069808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:41.069808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:41.069808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:41.069808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:41.069808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:41.069808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:41.069808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:41.069808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.069808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:41.069808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:41.069808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.069808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:41.070443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.070443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:41.071038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:41.071776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.072605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:41.072605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.072605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:41.072605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:41.072605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:41.072605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:41.072605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:28:41.072605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:41.073013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.073222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:41.073222 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.073222 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:41.073222 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:41.073222 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.073222 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:41.073222 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.073222 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:41.074424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:41.074909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:41.074909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:41.074909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.074909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:41.074909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:41.074909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.074909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:41.074909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.074909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:41.075753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.075753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:41.075753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:41.075753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:41.075753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:41.075753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:41.075753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:41.075753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:28:41.075753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:41.075753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:41.075753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.075753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:41.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:41.076528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.076528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:41.076528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.076528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:41.077575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:41.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:41.078758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.078758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:41.078758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:41.078758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.078758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:41.078758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.078758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:41.118588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:41.119782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:41.119991 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:41.120199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:41.120689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:41.120918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.121639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:41.121639 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.121639 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:41.121639 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:41.121639 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.121639 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:41.121639 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.121639 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:41.123114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:41.123114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:41.123114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.123114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:41.123114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:41.123114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.123114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:41.123114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.123114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:41.123821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.123821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:41.123821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:41.123821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:41.123821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:41.123821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:41.123821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:41.123821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:41.123821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:41.123821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.123821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:41.124249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:41.124648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.124648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:41.124648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.124648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:41.124926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:41.125457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:41.126002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.126002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:41.126002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.126002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:41.127913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:41.128352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:41.128463 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:41.128663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:41.129041 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:41.129270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:41.129392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.129947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:41.129947 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.129947 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:41.129947 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:41.129947 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.129947 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:41.129947 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.129947 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:41.131004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.131004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:41.131004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:41.131004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:41.131004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:41.131004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:41.131004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:41.131004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.131004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:41.131004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:41.131004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:41.131004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.131401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:41.131401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:41.131401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:41.131401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:41.131401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:41.131401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:41.131401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:41.131401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:41.131401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.131401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:41.131401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:41.131401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.131401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:41.131853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.131853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:41.132085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:41.132587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:41.133148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:41.133148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:41.133148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:45.605371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:45.606310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:45.606310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:45.607611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:45.642410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:45.642738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:45.643697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:45.643697 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:45.643697 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:45.643697 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:45.643697 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:45.643697 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:45.643697 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:45.643697 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:45.644764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:45.644764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:45.644764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:45.644764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:45.644764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:45.644764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:45.644764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:45.644764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:45.644764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:45.644764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:45.644764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:45.644764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:45.645338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:45.645338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:45.645338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:45.645338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:45.645338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:45.645338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:45.645338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:45.645338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:45.645338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:45.645338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:45.645338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:45.645338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:45.645338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:45.645862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:45.646520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:45.647358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:45.647358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:45.647358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:45.647358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:45.647358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:45.647358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:46.064964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:46.065428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:46.065621 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:46.066451 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:46.066744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.067595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:46.067595 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.067595 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:46.067595 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:46.067595 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.067595 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:46.067595 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.067595 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:46.068976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.068976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:46.068976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:46.068976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:46.068976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:46.068976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:46.068976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:46.068976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.068976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:46.068976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:46.068976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:46.068976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.069649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:46.069649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:46.069649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:46.069649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:46.069649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:46.069649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:46.069649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:46.069649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:46.069649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.069649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:46.069649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:46.069649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.069649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:46.070180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.070180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:46.070642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:46.071343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.072170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:46.072170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.072170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:46.072170 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:46.072170 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:46.072455 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:46.072516 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:28:46.072960 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:46.073362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.074024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:46.074024 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.074024 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:46.074024 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:46.074024 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.074024 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:46.074024 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.074024 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:46.075151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:46.075844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:46.075844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:46.075844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.075844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:46.075844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:46.075844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.075844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:46.075844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.075844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:46.076462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.076462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:46.076462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:46.076462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:46.076462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:46.076462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:46.076462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:46.076462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:28:46.076462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:46.076462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:46.076462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.076462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:46.076998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:46.077628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.077628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:46.077628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.077628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:46.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:46.078372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:46.079063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.079063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:46.079063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:46.079063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.079063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:46.079063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.079063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:46.119512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:46.119962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:46.120153 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:46.120432 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:46.120963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:46.121374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.122341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:46.122341 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.122341 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:46.122341 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:46.122341 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.122341 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:46.122341 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.122341 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:46.123851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:46.124500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:46.124500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:46.124500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.124500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:46.124500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:46.124500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.124500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:46.124500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.124500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:46.125255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.125255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:46.125255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:46.125255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:46.125255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:46.125255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:46.125255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:46.125255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:46.125255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:46.125255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.125255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:46.125735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:46.126353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.126353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:46.126353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.126353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:46.127074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:46.127837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:46.128619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.128619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:46.128619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.128619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:46.128619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:46.128870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:46.128870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:46.129057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:46.129504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:46.129832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:46.130084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.130714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:46.130714 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.130714 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:46.130714 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:46.130714 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.130714 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:46.130714 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.130714 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:46.131533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.131533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:46.131533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:46.131533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:46.131533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:46.131533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:46.131533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:46.131533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.131533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:46.131533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:46.131533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:46.131533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:46.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:46.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:46.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:46.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:46.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:46.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:46.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:46.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:46.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:46.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:46.132627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:46.133282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:46.134123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:46.134123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:46.134123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:46.134123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:46.134123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:46.134123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:50.606093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:50.607128 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:50.607294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:50.608709 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:50.639711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:50.640142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:50.641128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:50.641128 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:50.641128 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:50.641128 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:50.641128 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:50.641128 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:50.641128 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:50.641128 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:50.642046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:50.642046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:50.642046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:50.642046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:50.642046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:50.642046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:50.642046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:50.642046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:50.642046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:50.642046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:50.642046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:50.642046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:50.642702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:50.642702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:50.642702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:50.642702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:50.642702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:50.642702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:50.642702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:50.642702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:50.642702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:50.642702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:50.642702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:50.642702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:50.642702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:50.643263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:50.643899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:50.644765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:50.644765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:50.644765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:50.644765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:50.644765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:50.644765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:51.065667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:51.066540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:51.066820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:51.068083 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:51.068502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.069573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:51.069573 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.069573 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:51.069573 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:51.069573 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.069573 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:51.069573 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.069573 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:51.070673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.070673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:51.070673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:51.070673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:51.070673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:51.070673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:51.070673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:51.070673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.070673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:51.070673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:51.070673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:51.070673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.071299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:51.071299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:51.071299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:51.071299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:51.071299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:51.071299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:51.071299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:51.071299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:51.071299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.071299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:51.071299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:51.071299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.071299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:51.071847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:51.072545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.073430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:51.073430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:51.073430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.073430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:51.073430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.073430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:51.073430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:51.073430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:51.073430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:51.073430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:28:51.073983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:51.073983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.074102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:51.074102 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.074102 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:51.074102 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:51.074102 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.074102 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:51.074102 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.074102 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:51.074936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:51.075950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:51.076804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:51.076804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:51.076804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:28:51.076804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:51.076804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:51.076804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.076804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:51.076804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:51.076804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:51.076804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.076804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:51.076804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:51.076804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:51.077517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:51.077517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:51.077517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:51.077517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:51.077517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:51.077517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.077517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:51.077517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:51.077517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.077517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:51.077517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.077517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:51.078301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:51.079093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:51.079933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.079933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:51.079933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:51.079933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.079933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:51.079933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.079933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:51.120130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:51.120459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:51.120566 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:51.120718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:51.121287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:51.121674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:51.122385 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.122385 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:51.122385 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:51.122385 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.122385 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:51.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.122385 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:51.123228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:51.123922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:51.123922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:51.123922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.123922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:51.123922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:51.123922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.123922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:51.123922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.123922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:51.124338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.124338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:51.124338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:51.124338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:51.124338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:51.124338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:51.124338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:51.124338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:51.124338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:51.124338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.124338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:51.124841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:51.125440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:51.126090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:51.126911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:51.126911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:51.126911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:51.126911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:51.126911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.126911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:51.126911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:51.126911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.126911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:51.126911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.126911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:51.128654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:51.128964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:51.129129 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:51.129423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:51.129855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:51.130172 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:51.130409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.131031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:51.131031 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.131031 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:51.131031 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:51.131031 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.131031 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:51.131031 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.131031 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:51.131902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.131902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:51.131902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:51.131902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:51.131902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:51.131902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:51.131902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:51.131902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.131902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:51.131902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:51.131902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:51.131902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.132258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:51.132258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:51.132258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:51.132258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:51.132258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:51.132258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:51.132258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:51.132258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:51.132258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.132258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:51.132258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:51.132258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.132258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:51.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:51.132897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:51.133338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:51.133870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:51.133870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:51.133870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:55.606897 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:55.607886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:55.608045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:55.609441 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:55.644250 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:55.644772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:55.645772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:55.645772 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:55.645772 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:55.645772 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:55.645772 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:55.645772 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:55.645772 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:55.645772 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:55.646668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:55.646668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:55.646668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:55.646668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:55.646668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:55.646668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:55.646668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:55.646668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:55.646668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:55.646668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:55.646668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:55.646668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:55.647148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:55.647148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:55.647148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:55.647148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:55.647148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:55.647148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:55.647148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:55.647148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:55.647148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:55.647148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:55.647148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:55.647148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:55.647148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:55.647812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:55.648698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:55.650102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:55.650102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:55.650102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:55.650102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:55.650102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:55.650102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:56.066405 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:56.067148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:56.067442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:56.068667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:56.069068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.070050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:56.070050 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.070050 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:56.070050 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:56.070050 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.070050 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:56.070050 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.070050 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:56.072132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.072132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:56.072132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:56.072132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:56.072132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:56.072132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:56.072132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:56.072132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.072132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:56.072132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:56.072132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:56.072132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.072748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:56.072748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:56.072748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:56.072748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:56.072748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:56.072748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:56.072748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:56.072748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:56.072748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.072748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:56.072748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:56.072748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.072748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:56.073398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.073398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:56.074187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:56.074733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.075316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:56.075316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.075316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:56.075316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:56.075316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:56.075316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:56.075316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:28:56.075874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:56.076218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.077102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:56.077102 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.077102 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:56.077102 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:56.077102 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.077102 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:56.077102 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.077102 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:56.078411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:56.078891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:56.078891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:56.078891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.078891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:56.078891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:56.078891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.078891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:56.078891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.078891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:56.079780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.079780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:56.079780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:56.079780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:56.079780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:56.079780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:56.079780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:56.079780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:28:56.079780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:56.079780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:56.079780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.079780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:56.080154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:56.080577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.080577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:56.080577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.080577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:56.081541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:56.082020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:56.082683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.082683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:56.082683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:56.082683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.082683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:56.082683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.082683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:56.120684 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:56.121098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:56.121345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:56.121606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:56.122561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:56.123050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.124005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:56.124005 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.124005 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:56.124005 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:56.124005 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.124005 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:56.124005 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.124005 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:28:56.125229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:28:56.125910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:56.125910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:28:56.125910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.125910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:28:56.125910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:28:56.125910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.125910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:28:56.125910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.125910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:28:56.126737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.126737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:56.126737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:56.126737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:56.126737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:56.126737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:56.126737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:56.126737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:56.126737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:56.126737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.126737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:56.127307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:56.127912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.127912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:56.127912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.127912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:56.128446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:56.129326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:56.130081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.130081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:56.130081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.130081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:28:56.130081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:28:56.130081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:28:56.130081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:28:56.130081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:28:56.130397 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:56.130756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:28:56.130947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.131693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:28:56.131693 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.131693 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:28:56.131693 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:28:56.131693 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.131693 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:28:56.131693 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.131693 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:28:56.132563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.132563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:28:56.132563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:56.132563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:28:56.132563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:28:56.132563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:56.132563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:56.132563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.132563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:56.132563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:56.132563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:28:56.132563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.133080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:28:56.133080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:56.133080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:28:56.133080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:56.133080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:28:56.133080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:28:56.133080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:56.133080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:28:56.133080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.133080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:28:56.133080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:28:56.133080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.133080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:28:56.133470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.133470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:28:56.133679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:28:56.134163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:28:56.134751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:28:56.134751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:28:56.134751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:00.607654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:00.608680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:00.608680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:00.610259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:00.645504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:00.645954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:00.647037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:00.647037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:00.647037 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:00.647037 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:00.647037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:00.647037 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:00.647037 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:00.647037 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:00.648324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:00.648324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:00.648324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:00.648324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:00.648324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:00.648324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:00.648324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:00.648324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:00.648324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:00.648324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:00.648324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:00.648324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:00.649251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:00.649251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:00.649251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:00.649251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:00.649251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:00.649251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:00.649251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:00.649251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:00.649251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:00.649251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:00.649251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:00.649251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:00.649251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:00.649775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:00.650572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:00.652007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:00.652007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:00.652007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:00.652007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:00.652007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:00.652007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:01.067069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:01.068016 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:01.068177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:01.069309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:01.069673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.070660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:01.070660 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.070660 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:01.070660 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:01.070660 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.070660 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:01.070660 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.070660 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:01.072499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.072499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:01.072499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:01.072499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:01.072499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:01.072499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:01.072499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:01.072499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.072499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:01.072499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:01.072499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:01.072499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.073378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:01.073378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:01.073378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:01.073378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:01.073378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:01.073378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:01.073378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:01.073378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:01.073378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.073378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:01.073378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:01.073378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.073378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:01.074538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:01.075911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.076576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:01.076576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:01.076576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.076576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:01.076576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.076576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:01.076576 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:01.076576 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:01.076576 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:01.076576 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:29:01.077309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:01.077309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.077309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:01.077309 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.077309 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:01.077309 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:01.077309 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.077309 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:01.077309 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.077309 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:01.078287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:01.078780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:01.078780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:01.078780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.078780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:01.078780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:01.078780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.078780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:01.078780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.078780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:01.079192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.079192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:01.079192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:01.079192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:01.079192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:01.079192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:01.079192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:01.079192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:29:01.079192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:01.079192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:01.079192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.079192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:01.079582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:01.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:01.081002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:01.081500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:01.081500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:01.081500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:01.081500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:01.081500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:01.081500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.081500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:01.081500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:01.081500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.081500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:01.081500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.081500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:01.121261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:01.122236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:01.122462 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:01.122715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:01.123428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:01.123803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.124648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:01.124648 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.124648 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:01.124648 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:01.124648 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.124648 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:01.124648 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.124648 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:01.126242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:01.126842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:01.126842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:01.126842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.126842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:01.126842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:01.126842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.126842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:01.126842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.126842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:01.127957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.127957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:01.127957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:01.127957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:01.127957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:01.127957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:01.127957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:01.127957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:01.127957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:01.127957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.127957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:01.128501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:01.129030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.129030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:01.129030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.129030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:01.129194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:01.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:01.130615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.130615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:01.130615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.130615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:01.130615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:01.130615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:01.130615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:01.130615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:01.131063 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:01.131063 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:01.131063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.132023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:01.132023 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.132023 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:01.132023 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:01.132023 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.132023 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:01.132023 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.132023 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:01.132584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.132584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:01.132584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:01.132584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:01.132584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:01.132584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:01.132584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:01.132584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.132584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:01.132584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:01.132584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:01.132584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:01.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:01.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:01.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:01.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:01.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:01.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:01.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:01.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:01.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:01.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:01.133962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:01.134776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:01.135928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:01.135928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:01.135928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:01.135928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:01.135928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:01.135928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:05.608367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:05.609983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:05.610174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:05.611490 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:05.639039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:05.639463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:05.641370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:05.641370 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:05.641370 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:05.641370 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:05.641370 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:05.641370 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:05.641370 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:05.641370 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:05.642228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:05.642228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:05.642228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:05.642228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:05.642228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:05.642228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:05.642228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:05.642228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:05.642228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:05.642228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:05.642228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:05.642228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:05.642818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:05.642818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:05.642818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:05.642818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:05.642818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:05.642818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:05.642818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:05.642818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:05.642818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:05.642818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:05.642818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:05.642818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:05.642818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:05.643385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:05.643385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:05.644828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:05.645665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:05.647131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:05.647131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:05.647131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:06.067897 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:06.068478 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:06.068669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:06.069537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:06.069842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.070719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:06.070719 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.070719 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:06.070719 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:06.070719 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.070719 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:06.070719 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.070719 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:06.072225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.072225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:06.072225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:06.072225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:06.072225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:06.072225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:06.072225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:06.072225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.072225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:06.072225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:06.072225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:06.072225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.072809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:06.072809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:06.072809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:06.072809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:06.072809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:06.072809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:06.072809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:06.072809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:06.072809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.072809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:06.072809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:06.072809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.072809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:06.073394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.073394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:06.073832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:06.075476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.076642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:06.076642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.076642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:06.076642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:06.076642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:06.076642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:06.076642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:29:06.076642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:06.076889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.077388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:06.077388 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.077388 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:06.077388 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:06.077388 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.077388 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:06.077388 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.077388 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:06.078147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:06.079007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:06.079425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:06.079425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:06.079425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:29:06.079425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:06.079425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:06.079425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.079425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:06.079425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:06.079425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:06.079425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.079425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:06.079425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:06.079425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:06.079752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:06.079752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:06.079752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:06.079752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:06.079752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:06.079752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.079752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:06.079752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:06.079752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.079752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:06.079752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.079752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:06.080376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:06.080799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:06.081324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.081324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:06.081324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:06.081324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.081324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:06.081324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.081324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:06.121885 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:06.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:06.122474 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:06.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:06.123046 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:06.123577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.124113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:06.124113 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.124113 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:06.124113 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:06.124113 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.124113 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:06.124113 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.124113 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:06.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:06.126021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:06.126021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:06.126021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.126021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:06.126021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:06.126021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.126021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:06.126021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.126021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:06.126702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.126702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:06.126702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:06.126702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:06.126702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:06.126702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:06.126702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:06.126702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:06.126702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:06.126702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.126702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:06.127099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:06.127556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.127556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:06.127556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.127556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:06.128412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:06.128947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:06.129508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.129508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:06.129508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.129508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:06.129773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:06.130094 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:06.130294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:06.130518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:06.130828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:06.131051 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:06.131172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.131765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:06.131765 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.131765 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:06.131765 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:06.131765 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.131765 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:06.131765 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.131765 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:06.132689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.132689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:06.132689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:06.132689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:06.132689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:06.132689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:06.132689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:06.132689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.132689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:06.132689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:06.132689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:06.132689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.133062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:06.133062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:06.133062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:06.133062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:06.133062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:06.133062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:06.133062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:06.133062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:06.133062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.133062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:06.133062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:06.133062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.133062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:06.133455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.133455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:06.133716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:06.135198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:06.135933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:06.135933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:06.135933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:10.610025 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:10.610843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:10.610843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:10.612242 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:10.646093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:10.646558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:10.647422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:10.647422 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:10.647422 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:10.647422 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:10.647422 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:10.647422 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:10.647422 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:10.647422 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:10.648183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:10.648183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:10.648183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:10.648183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:10.648183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:10.648183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:10.648183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:10.648183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:10.648183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:10.648183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:10.648183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:10.648183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:10.648563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:10.648563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:10.648563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:10.648563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:10.648563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:10.648563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:10.648563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:10.648563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:10.648563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:10.648563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:10.648563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:10.648563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:10.648563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:10.648876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:10.648876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:10.649342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:10.651060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:10.651840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:10.651840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:10.651840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:11.068588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:11.069310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:11.069530 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:11.070454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:11.070778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.071638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:11.071638 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.071638 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:11.071638 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:11.071638 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.071638 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:11.071638 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.071638 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:11.073363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.073363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:11.073363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:11.073363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:11.073363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:11.073363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:11.073363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:11.073363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.073363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:11.073363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:11.073363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:11.073363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.073912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:11.073912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:11.073912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:11.073912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:11.073912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:11.073912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:11.073912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:11.073912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:11.073912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.073912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:11.073912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:11.073912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.073912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:11.074536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.074536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:11.075131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:11.075924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.077034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:11.077034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.077034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:11.077034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:11.077034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:11.077034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:11.077034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:29:11.077034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:11.077480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.078364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:11.078364 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.078364 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:11.078364 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:11.078364 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.078364 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:11.078364 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.078364 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:11.079593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:11.080009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:11.080009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:11.080009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.080009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:11.080009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:11.080009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.080009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:11.080009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.080009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:11.080957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.080957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:11.080957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:11.080957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:11.080957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:11.080957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:11.080957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:11.080957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:29:11.080957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:11.080957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:11.080957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.080957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:11.081663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:11.082338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:11.083004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:11.083951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:11.083951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:11.083951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:11.083951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:11.083951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:11.083951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.083951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:11.083951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:11.083951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.083951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:11.083951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.083951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:11.122363 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:11.122830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:11.123014 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:11.123232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:11.123831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:11.124318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.125157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:11.125157 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.125157 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:11.125157 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:11.125157 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.125157 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:11.125157 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.125157 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:11.126341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:11.126968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:11.126968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:11.126968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.126968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:11.126968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:11.126968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.126968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:11.126968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.126968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:11.127719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.127719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:11.127719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:11.127719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:11.127719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:11.127719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:11.127719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:11.127719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:11.127719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:11.127719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.127719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:11.128186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:11.129253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.129253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:11.129253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.129253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:11.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:11.130445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:11.130997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.130997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:11.130997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.130997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:11.130997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:11.130997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:11.130997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:11.131218 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:11.131584 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:11.131843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:11.131999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.132577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:11.132577 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.132577 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:11.132577 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:11.132577 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.132577 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:11.132577 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.132577 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:11.133550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.133550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:11.133550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:11.133550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:11.133550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:11.133550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:11.133550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:11.133550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.133550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:11.133550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:11.133550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:11.133550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.133914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:11.133914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:11.133914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:11.133914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:11.133914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:11.133914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:11.133914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:11.133914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:11.133914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.133914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:11.133914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:11.133914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.133914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:11.134280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.134280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:11.134799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:11.135254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:11.135760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:11.135760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:11.135760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:15.610685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:15.611495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:15.611495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:15.612662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:15.647414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:15.647932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:15.648974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:15.648974 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:15.648974 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:15.648974 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:15.648974 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:15.648974 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:15.648974 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:15.648974 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:15.650058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:15.650058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:15.650058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:15.650058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:15.650058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:15.650058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:15.650058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:15.650058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:15.650058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:15.650058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:15.650058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:15.650058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:15.650943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:15.650943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:15.650943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:15.650943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:15.650943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:15.650943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:15.650943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:15.650943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:15.650943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:15.650943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:15.650943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:15.650943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:15.650943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:15.651517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:15.652119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:15.652868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:15.652868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:15.652868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:15.652868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:15.652868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:15.652868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:16.069575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:16.070003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:16.070139 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:16.071248 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:16.071590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:16.072393 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.072393 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:16.072393 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:16.072393 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.072393 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:16.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.072393 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:16.073525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.073525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:16.073525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:16.073525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:16.073525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:16.073525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:16.073525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:16.073525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.073525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:16.073525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:16.073525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:16.073525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.074177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:16.074177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:16.074177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:16.074177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:16.074177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:16.074177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:16.074177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:16.074177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:16.074177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.074177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:16.074177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:16.074177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.074177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:16.074740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:16.076250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.077603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:16.077603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:16.077603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.077603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:16.077603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.077603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:16.077603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:16.077603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:16.077603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:16.077603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:29:16.078518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:16.078518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.078518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:16.078518 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.078518 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:16.078518 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:16.078518 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.078518 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:16.078518 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.078518 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:16.078933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:16.079790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:16.081107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:16.081107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:16.081107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:29:16.081107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:16.081107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:16.081107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.081107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:16.081107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:16.081107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:16.081107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.081107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:16.081107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:16.081107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:16.082148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:16.082148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:16.082148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:16.082148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:16.082148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:16.082148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.082148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:16.082148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:16.082148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.082148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:16.082148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.082148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:16.082485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:16.082766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:16.083060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.083060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:16.083060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:16.083060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.083060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:16.083060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.083060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:16.122937 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:16.123362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:16.123542 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:16.123738 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:16.124561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:16.124939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.126031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:16.126031 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.126031 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:16.126031 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:16.126031 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.126031 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:16.126031 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.126031 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:16.127348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:16.128386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:16.128386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:16.128386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.128386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:16.128386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:16.128386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.128386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:16.128386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.128386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:16.128985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.128985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:16.128985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:16.128985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:16.128985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:16.128985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:16.128985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:16.128985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:16.128985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:16.128985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.128985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:16.130101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:16.130808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:16.131377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:16.132118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:16.132118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:16.132118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:16.132118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:16.132118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.132118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:16.132118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:16.132118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.132118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:16.132118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.132118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:16.132118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:16.132118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:16.133961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:16.133961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:16.133961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:16.133961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:16.133961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.133961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:16.133961 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.133961 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:16.133961 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:16.134614 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.134614 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:16.134614 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.134614 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:16.134864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.134864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:16.134864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:16.134864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:16.134864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:16.134864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:16.134864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:16.134864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.134864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:16.134864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:16.134864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:16.134864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.135092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:16.135092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:16.135092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:16.135092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:16.135092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:16.135092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:16.135092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:16.135092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:16.135092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.135092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:16.135092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:16.135092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.135092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:16.135342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.135342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:16.136028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:16.136345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:16.136680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:16.136680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:16.136680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:20.611493 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:20.612650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:20.612650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:20.614126 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:20.651664 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:20.652142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:20.653224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:20.653224 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:20.653224 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:20.653224 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:20.653224 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:20.653224 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:20.653224 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:20.653224 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:20.654232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:20.654232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:20.654232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:20.654232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:20.654232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:20.654232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:20.654232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:20.654232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:20.654232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:20.654232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:20.654232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:20.654232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:20.654885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:20.654885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:20.654885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:20.654885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:20.654885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:20.654885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:20.654885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:20.654885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:20.654885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:20.654885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:20.654885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:20.654885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:20.654885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:20.655603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:20.656432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:20.657428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:20.657428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:20.657428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:20.657428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:20.657428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:20.657428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:21.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:21.071131 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:21.071383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:21.072689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:21.073090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.074093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:21.074093 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.074093 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:21.074093 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:21.074093 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.074093 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:21.074093 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.074093 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:21.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:21.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:21.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:21.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:21.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:21.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:21.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:21.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:21.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:21.075941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.077242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:21.077242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:21.077242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:21.077242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:21.077242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:21.077242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:21.077242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:21.077242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:21.077242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.077242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:21.077242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:21.077242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.077242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:21.077755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.077755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:21.078185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:21.079610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.080430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:21.080430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.080430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:21.080430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:21.080430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:21.080430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:21.080430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:29:21.080430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:21.080827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.080827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:21.080827 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.080827 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:21.080827 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:21.080827 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.080827 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:21.080827 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.080827 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:21.081836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:21.083147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:21.083147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:21.083147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.083147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:21.083147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:21.083147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.083147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:21.083147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.083147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:21.083751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.083751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:21.083751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:21.083751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:21.083751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:21.083751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:21.083751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:21.083751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:29:21.083751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:21.083751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:21.083751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.083751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:21.084189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:21.084649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:21.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:21.085870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:21.085870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:21.085870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:21.085870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:21.085870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:21.085870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.085870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:21.085870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:21.085870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.085870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:21.085870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.085870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:21.123960 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:21.124748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:21.124913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:21.125156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:21.126350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:21.126350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.127084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:21.127084 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.127084 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:21.127084 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:21.127084 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.127084 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:21.127084 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.127084 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:21.128483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:21.129186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:21.129186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:21.129186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.129186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:21.129186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:21.129186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.129186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:21.129186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.129186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:21.129837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.129837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:21.129837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:21.129837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:21.129837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:21.129837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:21.129837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:21.129837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:21.129837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:21.129837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.129837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:21.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:21.131001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.131001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:21.131001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.131001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:21.131874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:21.132688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:21.133549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.133549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:21.133549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.133549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:21.133549 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:21.133549 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:21.133549 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:21.133549 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:21.133950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:21.133950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:21.134067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.134550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:21.134550 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.134550 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:21.134550 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:21.134550 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.134550 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:21.134550 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.134550 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:21.135393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.135393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:21.135393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:21.135393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:21.135393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:21.135393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:21.135393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:21.135393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.135393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:21.135393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:21.135393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:21.135393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.136919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:21.136919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:21.136919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:21.136919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:21.136919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:21.136919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:21.136919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:21.136919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:21.136919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.136919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:21.136919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:21.136919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.136919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:21.137748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:21.138718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:21.140400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:21.140400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:21.140400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:21.140400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:21.140400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:21.140400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:25.612482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:25.613397 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:25.613588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:25.614971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:25.650552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:25.651005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:25.651995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:25.651995 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:25.651995 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:25.651995 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:25.651995 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:25.651995 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:25.651995 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:25.651995 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:25.652930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:25.652930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:25.652930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:25.652930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:25.652930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:25.652930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:25.652930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:25.652930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:25.652930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:25.652930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:25.652930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:25.652930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:25.653533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:25.653533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:25.653533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:25.653533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:25.653533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:25.653533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:25.653533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:25.653533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:25.653533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:25.653533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:25.653533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:25.653533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:25.653533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:25.654055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:25.654713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:25.655548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:25.655548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:25.655548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:25.655548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:25.655548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:25.655548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:26.071280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:26.072035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:26.072196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:26.073339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:26.073676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.074622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:26.074622 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.074622 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:26.074622 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:26.074622 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.074622 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:26.074622 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.074622 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:26.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:26.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:26.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:26.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:26.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:26.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:26.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:26.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:26.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:26.076128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.076730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:26.076730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:26.076730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:26.076730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:26.076730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:26.076730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:26.076730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:26.076730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:26.076730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.076730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:26.076730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:26.076730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.076730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:26.077243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.077243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:26.077763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:26.078452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.079156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:26.079156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.079156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:26.079156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:26.079156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:26.079418 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:26.079418 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:29:26.080004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:26.080356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.080927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:26.080927 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.080927 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:26.080927 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:26.080927 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.080927 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:26.080927 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.080927 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:26.081726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:26.083052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:26.084282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:26.084282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:26.084282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:29:26.084282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:26.084282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:26.084282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.084282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:26.084282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:26.084282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:26.084282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.084282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:26.084282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:26.084282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:26.085172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:26.085172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:26.085172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:26.085172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:26.085172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:26.085172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.085172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:26.085172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:26.085172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.085172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:26.085172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.085172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:26.085634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:26.086019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:26.086548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.086548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:26.086548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:26.086548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.086548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:26.086548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.086548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:26.124795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:26.125414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:26.125609 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:26.125793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:26.126330 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:26.126680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.127475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:26.127475 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.127475 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:26.127475 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:26.127475 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.127475 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:26.127475 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.127475 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:26.128763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:26.130045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:26.130045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:26.130045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.130045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:26.130045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:26.130045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.130045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:26.130045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.130045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:26.130739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.130739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:26.130739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:26.130739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:26.130739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:26.130739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:26.130739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:26.130739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:26.130739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:26.130739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.130739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:26.131194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:26.131964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:26.133157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:26.133792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:26.133792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:26.133792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:26.133792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:26.133792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.133792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:26.133792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:26.133792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.133792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:26.133792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.133792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:26.133792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:26.133792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:26.134127 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:26.134127 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:26.134127 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:26.134627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:26.134712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.135609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:26.135609 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.135609 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:26.135609 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:26.135609 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.135609 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:26.135609 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.135609 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:26.136476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.136476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:26.136476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:26.136476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:26.136476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:26.136476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:26.136476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:26.136476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.136476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:26.136476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:26.136476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:26.136476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.136861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:26.136861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:26.136861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:26.136861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:26.136861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:26.136861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:26.136861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:26.136861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:26.136861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.136861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:26.136861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:26.136861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.136861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:26.137579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:26.138400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:26.138911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:26.138911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:26.138911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:26.138911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:26.138911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:26.138911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:30.613755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:30.614819 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:30.614819 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:30.616033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:30.650688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:30.651426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:30.652502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:30.652502 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:30.652502 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:30.652502 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:30.652502 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:30.652502 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:30.652502 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:30.652502 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:30.654307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:30.654307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:30.654307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:30.654307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:30.654307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:30.654307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:30.654307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:30.654307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:30.654307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:30.654307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:30.654307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:30.654307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:30.655559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:30.655559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:30.655559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:30.655559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:30.655559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:30.655559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:30.655559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:30.655559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:30.655559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:30.655559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:30.655559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:30.655559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:30.655559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:30.656552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:30.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:30.658188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:30.658188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:30.658188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:30.658188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:30.658188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:30.658188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:31.072354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:31.073329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:31.074421 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:31.075438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:31.076432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:31.079456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:31.080032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.080420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:31.080420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:31.080420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.080420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:31.080420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.080420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:31.080420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:31.081747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:31.082367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.082896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:31.082896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.082896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:31.083305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:31.083520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:31.083742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:29:31.084447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:31.084690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.086313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:31.087264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.087923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:31.087923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:31.087923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:31.087923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:31.087923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:31.087923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:31.087923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:29:31.087923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:31.087923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:31.087923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.087923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:31.087923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:31.087923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.088653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:31.089264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:31.089874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:31.091307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:31.091307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:31.091307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.091307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:31.091307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:31.091307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.091307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:31.091307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.091307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:31.125539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:31.126188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:31.126710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:31.127031 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:31.128337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:31.129370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.130113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:31.130113 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.130113 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:31.130113 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:31.130113 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.130113 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:31.130113 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.130113 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:31.133259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:31.134117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:31.134117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:31.134117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.134117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:31.134117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:31.134117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.134117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:31.134117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.134117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:31.135271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.135271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:31.135271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:31.135271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:31.135271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:31.135271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:31.135271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:31.135271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:31.135271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:31.135271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.135271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:31.136433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:31.137195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.137195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:31.137195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.137195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:31.137195 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:31.139678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:31.140197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:31.141342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.141342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:31.141342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.141342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:31.141342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:31.141342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:31.141342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:31.141342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:31.141894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:31.141894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.141894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:31.141894 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.141894 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:31.141894 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:31.141894 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.141894 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:31.141894 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.141894 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:31.143308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.143308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:31.143308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:31.143308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:31.143308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:31.143308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:31.143308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:31.143308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.143308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:31.143308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:31.143308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:31.143308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.144166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:31.144166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:31.144166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:31.144166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:31.144166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:31.144166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:31.144166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:31.144166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:31.144166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.144166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:31.144166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:31.144166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.144166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:31.144708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.144708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:31.144812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:31.145559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:31.147245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:31.147245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:31.147245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:35.614514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:35.615386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:35.615452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:35.616738 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:35.638346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=235 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:29:35.651944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:35.652170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:35.652747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:35.652747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:35.652747 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:35.652747 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:35.652747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:35.652747 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:35.652747 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:35.652747 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:35.653634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:35.653634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:35.653634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:35.653634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:35.653634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:35.653634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:35.653634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:35.653634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:35.653634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:35.653634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:35.653634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:35.653634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:35.654319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:35.654319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:35.654319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:35.654319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:35.654319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:35.654319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:35.654319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:35.654319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:35.654319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:35.654319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:35.654319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:35.654319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:35.654319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:35.655012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:35.655962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:35.657016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:35.657016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:35.657016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:35.657016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:35.657016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:35.657016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:36.073194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:36.074706 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:36.074865 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:36.075569 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:36.075744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.076365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:36.076365 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.076365 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:36.076365 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:36.076365 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.076365 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:36.076365 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.076365 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:36.077230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.077230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:36.077230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:36.077230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:36.077230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:36.077230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:36.077230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:36.077230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.077230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:36.077230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:36.077230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:36.077230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.077813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:36.077813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:36.077813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:36.077813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:36.077813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:36.077813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:36.077813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:36.077813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:36.077813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.077813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:36.077813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:36.077813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.077813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:36.078408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:36.079070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.079917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:36.079917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:36.079917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.079917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:36.079917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.079917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:36.083561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:36.083877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:36.083993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:36.084109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:29:36.084440 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:36.084624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.085090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:36.085090 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.085090 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:36.085090 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:36.085090 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.085090 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:36.085090 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.085090 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:36.086026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:36.086481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:36.086481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:36.086481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.086481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:36.086481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:36.086481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.086481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:36.086481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.086481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:36.087254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.087254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:36.087254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:36.087254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:36.087254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:36.087254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:36.087254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:36.087254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:29:36.087254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:36.087254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:36.087254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.087254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:36.087617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:36.087998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.087998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:36.087998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.087998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:36.088760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:36.089395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:36.089886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.089886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:36.089886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:36.089886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.089886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:36.089886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.089886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:36.126137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:36.126541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:36.126716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:36.126912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:36.127336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:36.127683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.128405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:36.128405 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.128405 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:36.128405 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:36.128405 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.128405 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:36.128405 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.128405 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:36.129498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:36.130120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:36.130120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:36.130120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.130120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:36.130120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:36.130120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.130120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:36.130120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.130120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:36.130874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.130874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:36.130874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:36.130874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:36.130874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:36.130874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:36.130874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:36.130874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:36.130874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:36.130874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.130874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:36.131412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:36.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:36.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.132068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:36.132690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:36.133692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:36.134568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.134568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:36.134568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.134568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:36.139335 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:36.139608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:36.139769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:36.139936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:36.140162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:36.140451 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:36.140566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.141076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:36.141076 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.141076 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:36.141076 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:36.141076 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.141076 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:36.141076 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.141076 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:36.142509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.142509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:36.142509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:36.142509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:36.142509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:36.142509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:36.142509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:36.142509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.142509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:36.142509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:36.142509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:36.142509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:36.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:36.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:36.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:36.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:36.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:36.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:36.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:36.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:36.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:36.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:36.143381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.143381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:36.144111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:36.144938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:36.145811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:36.145811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:36.145811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:40.615446 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:40.616320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:40.616320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:40.617513 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:40.652108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:40.652551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:40.653630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:40.653630 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:40.653630 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:40.653630 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:40.653630 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:40.653630 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:40.653630 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:40.653630 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:40.655725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:40.655725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:40.655725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:40.655725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:40.655725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:40.655725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:40.655725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:40.655725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:40.655725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:40.655725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:40.655725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:40.655725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:40.656227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:40.656227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:40.656227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:40.656227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:40.656227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:40.656227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:40.656227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:40.656227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:40.656227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:40.656227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:40.656227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:40.656227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:40.656227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:40.656858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:40.656858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:40.657617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:40.658380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:40.659198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:40.659198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:40.659198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:41.074778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:41.075611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:41.075819 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:41.076771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:41.077017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.077842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:41.077842 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.077842 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:41.077842 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:41.077842 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.077842 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:41.077842 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.077842 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:41.079399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.079399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:41.079399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:41.079399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:41.079399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:41.079399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:41.079399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:41.079399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.079399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:41.079399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:41.079399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:41.079399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.079938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:41.079938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:41.079938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:41.079938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:41.079938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:41.079938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:41.079938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:41.079938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:41.079938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.079938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:41.079938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:41.079938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.079938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:41.080523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.080523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:41.081025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:41.081826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.082636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:41.082636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.082636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:41.083985 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:41.084396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:41.084561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:41.084742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:29:41.085085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:41.085507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.086052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:41.086052 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.086052 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:41.086052 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:41.086052 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.086052 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:41.086052 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.086052 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:41.086918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:41.087392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:41.087392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:41.087392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.087392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:41.087392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:41.087392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.087392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:41.087392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.087392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:41.087690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.087690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:41.087690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:41.087690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:41.087690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:41.087690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:41.087690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:41.087690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:29:41.087690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:41.087690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:41.087690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.087690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:41.088047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:41.088461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.088461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:41.088461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.088461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:41.088688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:41.089151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:41.089737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.089737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:41.089737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:41.089737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.089737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:41.089737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.089737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:41.126656 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:41.127034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:41.127232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:41.127448 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:41.127846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:41.128024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.128581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:41.128581 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.128581 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:41.128581 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:41.128581 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.128581 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:41.128581 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.128581 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:41.129765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:41.130449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:41.130449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:41.130449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.130449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:41.130449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:41.130449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.130449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:41.130449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.130449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:41.131105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.131105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:41.131105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:41.131105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:41.131105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:41.131105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:41.131105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:41.131105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:41.131105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:41.131105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.131105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:41.131685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:41.132289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.132289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:41.132289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.132289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:41.132760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:41.133737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:41.134496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.134496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:41.134496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.134496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:41.139770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:41.140108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:41.140227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:41.140384 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:41.140678 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:41.140896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:41.141007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.141521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:41.141521 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.141521 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:41.141521 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:41.141521 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.141521 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:41.141521 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.141521 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:41.142349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.142349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:41.142349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:41.142349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:41.142349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:41.142349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:41.142349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:41.142349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.142349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:41.142349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:41.142349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:41.142349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:41.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:41.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:41.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:41.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:41.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:41.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:41.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:41.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:41.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:41.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:41.143156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.143156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:41.143261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:41.143732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:41.144307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:41.144307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:41.144307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:45.616110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:45.617073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:45.617073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:45.618079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:45.651842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:45.652915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:45.653340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:45.653340 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:45.653340 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:45.653340 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:45.653340 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:45.653340 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:45.653340 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:45.653340 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:45.654263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:45.654263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:45.654263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:45.654263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:45.654263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:45.654263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:45.654263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:45.654263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:45.654263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:45.654263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:45.654263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:45.654263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:45.654794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:45.654794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:45.654794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:45.654794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:45.654794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:45.654794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:45.654794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:45.654794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:45.654794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:45.654794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:45.654794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:45.654794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:45.654794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:45.655195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:45.655195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:45.657886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:45.658880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:45.659789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:45.659789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:45.659789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:46.075635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:46.076140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:46.077070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:46.077531 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:46.078399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.079277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:46.079277 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.079277 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:46.079277 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:46.079277 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.079277 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:46.079277 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.079277 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:46.080557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.080557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:46.080557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:46.080557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:46.080557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:46.080557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:46.080557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:46.080557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.080557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:46.080557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:46.080557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:46.080557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.081181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:46.081181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:46.081181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:46.081181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:46.081181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:46.081181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:46.081181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:46.081181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:46.081181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.081181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:46.081181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:46.081181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.081181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:46.081825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.081825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:46.083942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:46.085978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.086807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:46.086807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.086807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:46.086807 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:46.086807 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:46.086807 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:46.086807 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:29:46.086807 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:46.087167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.089337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:46.090074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.090755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:46.090755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:46.090755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:46.090755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:46.090755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:46.090755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:46.090755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:29:46.090755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:46.090755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:46.090755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.090755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:46.090755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:46.090755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.091253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:46.091804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:46.092364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:46.092692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:46.092692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:46.092692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.092692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:46.092692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:46.092692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.092692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:46.092692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.092692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:46.127114 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:46.127841 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:46.128035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:46.128272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:46.129148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:46.129587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.130515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:46.130515 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.130515 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:46.130515 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:46.130515 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.130515 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:46.130515 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.130515 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:46.131925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:46.132593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:46.132593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:46.132593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.132593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:46.132593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:46.132593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.132593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:46.132593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.132593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:46.133447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.133447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:46.133447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:46.133447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:46.133447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:46.133447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:46.133447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:46.133447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:46.133447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:46.133447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.133447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:46.134540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:46.135599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:46.136411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:46.136929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:46.136929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:46.136929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:46.136929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:46.136929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.136929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:46.136929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:46.136929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.136929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:46.136929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.136929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:46.140236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:46.140545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:46.140643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:46.140801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:46.141324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:46.141672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:46.141855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.142593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:46.142593 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.142593 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:46.142593 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:46.142593 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.142593 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:46.142593 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.142593 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:46.143465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.143465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:46.143465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:46.143465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:46.143465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:46.143465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:46.143465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:46.143465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.143465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:46.143465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:46.143465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:46.143465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:46.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:46.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:46.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:46.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:46.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:46.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:46.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:46.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:46.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:46.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:46.144095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.144095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:46.144436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:46.145510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:46.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:46.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:46.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:50.616856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:50.617944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:50.617944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:50.619111 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:50.646261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:50.646802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:50.647561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:50.647561 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:50.647561 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:50.647561 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:50.647561 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:50.647561 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:50.647561 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:50.647561 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:50.648465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:50.648465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:50.648465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:50.648465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:50.648465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:50.648465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:50.648465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:50.648465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:50.648465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:50.648465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:50.648465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:50.648465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:50.649035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:50.649035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:50.649035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:50.649035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:50.649035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:50.649035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:50.649035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:50.649035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:50.649035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:50.649035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:50.649035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:50.649035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:50.649035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:50.649585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:50.650416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:50.651941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:50.651941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:50.651941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:50.651941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:50.651941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:50.651941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:51.076273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:51.077015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:51.077161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:51.078051 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:51.078368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.079382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:51.079382 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.079382 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:51.079382 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:51.079382 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.079382 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:51.079382 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.079382 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:51.080796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.080796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:51.080796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:51.080796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:51.080796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:51.080796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:51.080796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:51.080796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.080796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:51.080796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:51.080796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:51.080796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.081405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:51.081405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:51.081405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:51.081405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:51.081405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:51.081405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:51.081405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:51.081405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:51.081405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.081405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:51.081405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:51.081405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.081405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:51.081930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.081930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:51.082571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:51.083309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.084088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:51.084088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.084088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:51.084883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:51.085110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:51.085239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:51.085418 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:29:51.085895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:51.086232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.086982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:51.086982 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.086982 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:51.086982 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:51.086982 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.086982 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:51.086982 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.086982 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:51.087693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:51.088126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:51.088126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:51.088126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.088126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:51.088126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:51.088126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.088126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:51.088126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.088126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:51.088503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.088503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:51.088503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:51.088503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:51.088503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:51.088503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:51.088503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:51.088503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:29:51.088503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:51.088503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:51.088503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.088503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:51.088855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:51.089240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.089240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:51.089240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.089240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:51.089513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:51.090046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:51.090561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.090561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:51.090561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:51.090561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.090561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:51.090561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.090561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:51.127889 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:51.128829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:51.129034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:51.129296 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:51.129969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:51.130461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.131356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:51.131356 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.131356 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:51.131356 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:51.131356 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.131356 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:51.131356 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.131356 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:51.132880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:51.133315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:51.133315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:51.133315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.133315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:51.133315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:51.133315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.133315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:51.133315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.133315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:51.134377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.134377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:51.134377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:51.134377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:51.134377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:51.134377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:51.134377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:51.134377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:51.134377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:51.134377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.134377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:51.134850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:51.135408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.135408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:51.135408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.135408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:51.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:51.137003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:51.137693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.137693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:51.137693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.137693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:51.140711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:51.141044 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:51.141172 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:51.141405 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:51.141892 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:51.142283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:51.142537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.143496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:51.143496 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.143496 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:51.143496 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:51.143496 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.143496 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:51.143496 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.143496 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:51.144464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.144464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:51.144464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:51.144464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:51.144464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:51.144464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:51.144464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:51.144464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.144464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:51.144464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:51.144464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:51.144464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.144945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:51.144945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:51.144945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:51.144945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:51.144945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:51.144945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:51.144945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:51.144945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:51.144945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.144945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:51.144945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:51.144945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.144945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:51.145457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.145457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:51.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:51.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:51.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:51.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:51.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:55.617551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:55.618272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:55.618343 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:55.619126 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:55.650876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:55.651215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:55.651954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:55.651954 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:55.651954 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:55.651954 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:55.651954 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:55.651954 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:55.651954 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:55.651954 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:55.652707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:55.652707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:55.652707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:55.652707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:55.652707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:55.652707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:55.652707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:55.652707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:55.652707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:55.652707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:55.652707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:55.652707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:55.653083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:55.653083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:55.653083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:55.653083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:55.653083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:55.653083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:55.653083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:55.653083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:55.653083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:55.653083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:55.653083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:55.653083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:55.653083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:55.653548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:55.653548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:55.653780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:55.654114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:55.654486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:55.654486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:55.654486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:56.077172 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:56.077743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:56.077966 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:56.078892 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:56.079224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.080099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:56.080099 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.080099 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:56.080099 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:56.080099 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.080099 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:56.080099 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.080099 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:56.081764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.081764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:56.081764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:56.081764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:56.081764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:56.081764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:56.081764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:56.081764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.081764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:56.081764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:56.081764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:56.081764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.082538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:56.082538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:56.082538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:56.082538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:56.082538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:56.082538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:56.082538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:56.082538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:56.082538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.082538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:56.082538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:56.082538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.082538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:56.083266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.083266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:56.083414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:56.084300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.085404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:56.085404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.085404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:56.085404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:56.085543 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:56.085661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:56.085773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:29:56.086096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:56.086337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.086858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:56.086858 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.086858 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:56.086858 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:56.086858 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.086858 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:56.086858 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.086858 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:56.087786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:56.088236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:56.088236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:56.088236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.088236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:56.088236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:56.088236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.088236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:56.088236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.088236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:56.088813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.088813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:56.088813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:56.088813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:56.088813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:56.088813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:56.088813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:56.088813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:29:56.088813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:56.088813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:56.088813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.088813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:56.089176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:56.089565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.089565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:56.089565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.089565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:56.090079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:56.090545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:56.091098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.091098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:56.091098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:56.091098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.091098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:56.091098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.091098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:56.128521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:56.128939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:56.129234 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:56.129517 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:56.130056 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:56.130478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.131285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:56.131285 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.131285 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:56.131285 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:56.131285 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.131285 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:56.131285 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.131285 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:29:56.132594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:29:56.133290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:56.133290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:29:56.133290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.133290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:29:56.133290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:29:56.133290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.133290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:29:56.133290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.133290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:29:56.134062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.134062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:56.134062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:56.134062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:56.134062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:56.134062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:56.134062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:56.134062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:56.134062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:56.134062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.134062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:56.134604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:56.135189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.135189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:56.135189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.135189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:56.135594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:56.136368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:56.137424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.137424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:56.137424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.137424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:29:56.141170 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:29:56.141498 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:29:56.141629 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:29:56.141820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:29:56.142291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:56.142671 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:29:56.142861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.143668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:29:56.143668 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.143668 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:29:56.143668 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:29:56.143668 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.143668 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:29:56.143668 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.143668 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:29:56.144737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.144737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:29:56.144737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:56.144737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:29:56.144737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:29:56.144737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:56.144737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:56.144737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.144737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:56.144737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:56.144737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:29:56.144737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.145113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:29:56.145113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:56.145113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:29:56.145113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:56.145113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:29:56.145113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:29:56.145113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:56.145113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:29:56.145113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.145113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:29:56.145113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:29:56.145113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.145113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:29:56.145507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.145507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:29:56.146012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:29:56.146491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:29:56.147328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:29:56.147328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:29:56.147328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:00.618240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:00.619102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:00.619102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:00.619752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:00.648661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:00.648882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:00.649689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:00.649689 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:00.649689 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:00.649689 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:00.649689 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:00.649689 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:00.649689 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:00.649689 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:00.650431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:00.650431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:00.650431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:00.650431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:00.650431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:00.650431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:00.650431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:00.650431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:00.650431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:00.650431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:00.650431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:00.650431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:00.651022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:00.651022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:00.651022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:00.651022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:00.651022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:00.651022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:00.651022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:00.651022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:00.651022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:00.651022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:00.651022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:00.651022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:00.651022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:00.651602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:00.652354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:00.653723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:00.653723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:00.653723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:00.653723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:00.653723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:00.653723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:01.077863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:01.078497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:01.078715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:01.079634 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:01.079957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.080853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:01.080853 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.080853 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:01.080853 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:01.080853 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.080853 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:01.080853 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.080853 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:01.082535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.082535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:01.082535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:01.082535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:01.082535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:01.082535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:01.082535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:01.082535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.082535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:01.082535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:01.082535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:01.082535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.083306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:01.083306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:01.083306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:01.083306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:01.083306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:01.083306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:01.083306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:01.083306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:01.083306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.083306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:01.083306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:01.083306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.083306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:01.083892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.083892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:01.084719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:01.085461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.086391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:01.086391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.086391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:01.086391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:01.086391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:01.086391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:01.086711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:30:01.086948 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:01.087238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.087907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:01.087907 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.087907 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:01.087907 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:01.087907 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.087907 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:01.087907 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.087907 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:01.088819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:01.089291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:01.089291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:01.089291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.089291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:01.089291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:01.089291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.089291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:01.089291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.089291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:01.089914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.089914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:01.089914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:01.089914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:01.089914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:01.089914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:01.089914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:01.089914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:30:01.089914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:01.089914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:01.089914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.089914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:01.090310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:01.090705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.090705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:01.090705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.090705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:01.091228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:01.091734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:01.092319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.092319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:01.092319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:01.092319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.092319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:01.092319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.092319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:01.128992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:01.129803 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:01.130078 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:01.130378 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:01.130818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:01.131050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.131640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:01.131640 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.131640 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:01.131640 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:01.131640 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.131640 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:01.131640 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.131640 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:01.132825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:01.133247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:01.133247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:01.133247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.133247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:01.133247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:01.133247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.133247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:01.133247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.133247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:01.134000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.134000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:01.134000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:01.134000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:01.134000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:01.134000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:01.134000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:01.134000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:01.134000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:01.134000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.134000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:01.134341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:01.135018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.135018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:01.135018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.135018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:01.135628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:01.136255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:01.136803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.136803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:01.136803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.136803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:01.141716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:01.142015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:01.142147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:01.142370 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:01.142804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:01.143136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:01.143363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.144103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:01.144103 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.144103 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:01.144103 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:01.144103 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.144103 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:01.144103 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.144103 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:01.145361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.145361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:01.145361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:01.145361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:01.145361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:01.145361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:01.145361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:01.145361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.145361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:01.145361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:01.145361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:01.145361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.146312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:01.146312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:01.146312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:01.146312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:01.146312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:01.146312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:01.146312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:01.146312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:01.146312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.146312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:01.146312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:01.146312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.146312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:01.147168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:01.148017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:01.149048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:01.149048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:01.149048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:01.149048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:01.149048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:01.149048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:05.618842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:05.619500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:05.619685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:05.620938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:05.655869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:05.656324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:05.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:05.657198 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:05.657198 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:05.657198 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:05.657198 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:05.657198 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:05.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:05.657198 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:05.658240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:05.658240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:05.658240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:05.658240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:05.658240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:05.658240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:05.658240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:05.658240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:05.658240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:05.658240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:05.658240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:05.658240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:05.658879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:05.658879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:05.658879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:05.658879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:05.658879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:05.658879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:05.658879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:05.658879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:05.658879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:05.658879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:05.658879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:05.658879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:05.658879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:05.659260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:05.659260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:05.659347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:05.659905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:05.660464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:05.660464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:05.660464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:06.078644 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:06.079505 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:06.079842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:06.080985 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:06.081333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.082227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:06.082227 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.082227 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:06.082227 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:06.082227 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.082227 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:06.082227 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.082227 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:06.084414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.084414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:06.084414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:06.084414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:06.084414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:06.084414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:06.084414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:06.084414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.084414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:06.084414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:06.084414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:06.084414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.084969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:06.084969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:06.084969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:06.084969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:06.084969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:06.084969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:06.084969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:06.084969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:06.084969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.084969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:06.084969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:06.084969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.084969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:06.085406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.085406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:06.086603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:06.087111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.087643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:06.087643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.087643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:06.087643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:06.087643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:06.087643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:06.087643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:30:06.088121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:06.088691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.089222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:06.089222 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.089222 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:06.089222 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:06.089222 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.089222 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:06.089222 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.089222 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:06.089955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:06.090382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:06.090382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:06.090382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.090382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:06.090382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:06.090382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.090382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:06.090382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.090382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:06.090784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.090784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:06.090784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:06.090784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:06.090784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:06.090784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:06.090784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:06.090784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:30:06.090784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:06.090784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:06.090784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.090784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:06.091350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:06.091689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.091689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:06.091689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.091689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:06.092704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:06.093151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:06.093746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.093746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:06.093746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:06.093746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.093746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:06.093746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.093746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:06.129545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:06.130768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:06.131008 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:06.131325 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:06.132363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.133381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:06.133381 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.133381 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:06.133381 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:06.133381 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.133381 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:06.133381 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.133381 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:06.134809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:06.135263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:06.135263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:06.135263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.135263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:06.135263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:06.135263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.135263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:06.135263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.135263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:06.136458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.136458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:06.136458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:06.136458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:06.136458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:06.136458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:06.136458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:06.136458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:06.136458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:06.136458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.136458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:06.136899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:06.137289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.137289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:06.137289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.137289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:06.138419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:06.139054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:06.139624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.139624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:06.139624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.139624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:06.142228 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:06.142611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:06.142731 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:06.143017 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:06.143544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:06.143878 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:06.144056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.144826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:06.144826 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.144826 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:06.144826 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:06.144826 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.144826 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:06.144826 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.144826 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:06.145893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.145893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:06.145893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:06.145893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:06.145893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:06.145893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:06.145893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:06.145893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.145893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:06.145893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:06.145893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:06.145893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.146247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:06.146247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:06.146247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:06.146247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:06.146247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:06.146247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:06.146247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:06.146247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:06.146247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.146247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:06.146247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:06.146247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.146247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:06.146568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.146568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:06.146915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:06.147368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:06.147834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:06.147834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:06.147834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:10.619565 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:10.620487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:10.620601 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:10.622070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:10.659826 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:10.659826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:10.660351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:10.660351 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:10.660351 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:10.660351 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:10.660351 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:10.660351 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:10.660351 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:10.660351 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:10.661585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:10.661585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:10.661585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:10.661585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:10.661585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:10.661585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:10.661585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:10.661585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:10.661585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:10.661585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:10.661585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:10.661585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:10.661953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:10.661953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:10.661953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:10.661953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:10.661953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:10.661953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:10.661953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:10.661953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:10.661953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:10.661953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:10.661953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:10.661953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:10.661953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:10.662345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:10.662345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:10.662840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:10.663290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:10.663783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:10.663783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:10.663783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:11.079373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:11.079984 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:11.080190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:11.081414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:11.081414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.082279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:11.082279 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.082279 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:11.082279 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:11.082279 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.082279 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:11.082279 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.082279 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:11.083824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.083824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:11.083824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:11.083824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:11.083824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:11.083824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:11.083824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:11.083824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.083824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:11.083824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:11.083824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:11.083824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.084550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:11.084550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:11.084550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:11.084550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:11.084550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:11.084550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:11.084550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:11.084550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:11.084550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.084550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:11.084550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:11.084550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.084550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:11.085278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.085278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:11.085737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:11.086624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.087559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:11.087559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.087559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:11.087559 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:11.087789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:11.087789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:11.087920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:30:11.088150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:11.088344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.088805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:11.088805 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.088805 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:11.088805 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:11.088805 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.088805 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:11.088805 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.088805 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:11.089566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:11.090086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:11.090086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:11.090086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.090086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:11.090086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:11.090086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.090086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:11.090086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.090086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:11.090429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.090429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:11.090429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:11.090429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:11.090429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:11.090429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:11.090429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:11.090429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:30:11.090429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:11.090429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:11.090429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.090429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:11.090766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:11.091134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.091134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:11.091134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.091134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:11.091412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:11.091885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:11.092361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.092361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:11.092361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:11.092361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.092361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:11.092361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.092361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:11.130328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:11.130978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:11.131171 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:11.131442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:11.132014 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:11.132447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.133244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:11.133244 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.133244 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:11.133244 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:11.133244 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.133244 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:11.133244 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.133244 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:11.134708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:11.135506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:11.135506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:11.135506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.135506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:11.135506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:11.135506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.135506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:11.135506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.135506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:11.136224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.136224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:11.136224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:11.136224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:11.136224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:11.136224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:11.136224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:11.136224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:11.136224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:11.136224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.136224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:11.136725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:11.137434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.137434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:11.137434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.137434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:11.137970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:11.138820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:11.139609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.139609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:11.139609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.139609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:11.142730 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:11.143003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:11.143110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:11.143303 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:11.143596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:11.143801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:11.143903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.144449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:11.144449 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.144449 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:11.144449 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:11.144449 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.144449 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:11.144449 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.144449 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:11.145454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.145454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:11.145454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:11.145454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:11.145454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:11.145454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:11.145454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:11.145454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.145454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:11.145454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:11.145454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:11.145454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.145915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:11.145915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:11.145915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:11.145915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:11.145915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:11.145915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:11.145915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:11.145915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:11.145915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.145915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:11.145915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:11.145915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.145915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:11.146615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:11.147488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:11.148176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:11.148176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:11.148176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:11.148176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:11.148176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:11.148176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:15.620273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:15.620769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:15.620916 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:15.622219 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:15.659704 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:15.659890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:15.660386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:15.660386 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:15.660386 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:15.660386 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:15.660386 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:15.660386 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:15.660386 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:15.660386 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:15.661340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:15.661340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:15.661340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:15.661340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:15.661340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:15.661340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:15.661340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:15.661340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:15.661340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:15.661340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:15.661340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:15.661340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:15.661627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:15.661627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:15.661627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:15.661627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:15.661627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:15.661627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:15.661627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:15.661627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:15.661627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:15.661627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:15.661627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:15.661627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:15.661627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:15.661889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:15.661889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:15.662456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:15.662776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:15.663147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:15.663147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:15.663147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:16.079914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:16.080146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:16.080316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:16.080919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:16.081180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.082080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:16.082080 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.082080 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:16.082080 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:16.082080 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.082080 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:16.082080 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.082080 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:16.083487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.083487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:16.083487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:16.083487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:16.083487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:16.083487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:16.083487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:16.083487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.083487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:16.083487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:16.083487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:16.083487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.084070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:16.084070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:16.084070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:16.084070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:16.084070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:16.084070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:16.084070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:16.084070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:16.084070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.084070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:16.084070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:16.084070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.084070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:16.084648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.084648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:16.087248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:16.088019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.089197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:16.089197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.089197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:16.089197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:16.089197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:16.089197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:16.089197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:30:16.089197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:16.089851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.089851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:16.089851 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.089851 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:16.089851 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:16.089851 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.089851 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:16.089851 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.089851 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:16.092150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:16.092771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:16.093375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:16.093375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:16.093375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:30:16.093375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:16.093375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:16.093375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.093375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:16.093375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:16.093375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:16.093375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.093375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:16.093375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:16.093375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:16.093863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:16.093863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:16.093863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:16.093863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:16.093863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:16.093863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.093863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:16.093863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:16.093863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.093863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:16.093863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.093863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:16.094366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:16.094942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:16.095674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.095674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:16.095674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:16.095674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.095674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:16.095674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.095674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:16.130878 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:16.131116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:16.131228 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:16.131374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:16.131841 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:16.132218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.132905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:16.132905 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.132905 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:16.132905 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:16.132905 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.132905 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:16.132905 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.132905 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:16.133705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:16.134402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:16.134402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:16.134402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.134402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:16.134402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:16.134402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.134402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:16.134402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.134402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:16.134772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.134772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:16.134772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:16.134772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:16.134772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:16.134772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:16.134772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:16.134772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:16.134772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:16.134772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.134772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:16.135293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:16.135877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:16.136538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:16.137515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:16.137515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:16.137515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:16.137515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:16.137515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.137515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:16.137515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:16.137515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.137515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:16.137515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.137515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:16.143018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:16.143203 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:16.143342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:16.143487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:16.143853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:16.144177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:16.144375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.144968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:16.144968 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.144968 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:16.144968 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:16.144968 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.144968 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:16.144968 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.144968 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:16.145820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.145820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:16.145820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:16.145820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:16.145820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:16.145820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:16.145820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:16.145820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.145820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:16.145820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:16.145820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:16.145820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.146167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:16.146167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:16.146167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:16.146167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:16.146167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:16.146167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:16.146167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:16.146167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:16.146167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.146167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:16.146167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:16.146167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.146167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:16.146521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.146521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:16.146795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:16.147233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:16.147806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:16.147806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:16.147806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:20.620990 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:20.621918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:20.621918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:20.622796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:20.651904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:20.652130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:20.652804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:20.652804 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:20.652804 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:20.652804 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:20.652804 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:20.652804 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:20.652804 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:20.652804 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:20.653574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:20.653574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:20.653574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:20.653574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:20.653574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:20.653574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:20.653574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:20.653574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:20.653574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:20.653574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:20.653574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:20.653574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:20.654129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:20.654129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:20.654129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:20.654129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:20.654129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:20.654129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:20.654129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:20.654129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:20.654129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:20.654129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:20.654129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:20.654129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:20.654129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:20.654529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:20.654953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:20.655511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:20.655511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:20.655511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:20.655511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:20.655511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:20.655511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:21.080720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:21.081734 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:21.081873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:21.082540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:21.082832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.083598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:21.083598 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.083598 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:21.083598 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:21.083598 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.083598 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:21.083598 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.083598 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:21.084503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.084503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:21.084503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:21.084503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:21.084503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:21.084503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:21.084503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:21.084503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.084503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:21.084503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:21.084503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:21.084503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.084829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:21.084829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:21.084829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:21.084829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:21.084829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:21.084829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:21.084829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:21.084829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:21.084829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.084829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:21.084829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:21.084829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.084829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:21.085160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.085160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:21.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:21.085973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.086469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:21.086469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.086469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:21.088038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:21.088349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:21.088458 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:21.088573 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:30:21.088840 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:21.089072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.089654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:21.089654 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.089654 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:21.089654 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:21.089654 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.089654 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:21.089654 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.089654 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:21.090362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:21.090749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:21.090749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:21.090749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.090749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:21.090749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:21.090749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.090749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:21.090749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.090749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:21.091225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.091225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:21.091225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:21.091225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:21.091225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:21.091225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:21.091225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:21.091225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:30:21.091225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:21.091225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:21.091225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.091225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:21.091626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:21.092051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.092051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:21.092051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.092051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:21.092997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:21.093510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:21.093968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.093968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:21.093968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:21.093968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.093968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:21.093968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.093968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:21.131326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:21.131738 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:21.131928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:21.132122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:21.132637 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:21.133069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.133928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:21.133928 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.133928 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:21.133928 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:21.133928 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.133928 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:21.133928 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.133928 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:21.135074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:21.136019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:21.136019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:21.136019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.136019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:21.136019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:21.136019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.136019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:21.136019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.136019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:21.136604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.136604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:21.136604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:21.136604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:21.136604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:21.136604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:21.136604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:21.136604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:21.136604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:21.136604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.136604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:21.137324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:21.138104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:21.138876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:21.139996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:21.139996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:21.139996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:21.139996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:21.139996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.139996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:21.139996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:21.139996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.139996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:21.139996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.139996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:21.143454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:21.143725 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:21.143831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:21.143976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:21.144270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:21.144504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:21.144634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.145091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:21.145091 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.145091 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:21.145091 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:21.145091 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.145091 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:21.145091 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.145091 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:21.145861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.145861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:21.145861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:21.145861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:21.145861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:21.145861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:21.145861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:21.145861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.145861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:21.145861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:21.145861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:21.145861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.146319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:21.146319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:21.146319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:21.146319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:21.146319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:21.146319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:21.146319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:21.146319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:21.146319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.146319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:21.146319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:21.146319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.146319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:21.146699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.146699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:21.146809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:21.147238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:21.147851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:21.147851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:21.147851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:25.621698 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:25.622466 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:25.622466 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:25.623008 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:25.655241 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:25.655652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:25.656554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:25.656554 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:25.656554 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:25.656554 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:25.656554 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:25.656554 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:25.656554 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:25.656554 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:25.657271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:25.657271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:25.657271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:25.657271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:25.657271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:25.657271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:25.657271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:25.657271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:25.657271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:25.657271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:25.657271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:25.657271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:25.657874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:25.657874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:25.657874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:25.657874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:25.657874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:25.657874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:25.657874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:25.657874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:25.657874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:25.657874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:25.657874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:25.657874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:25.657874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:25.658493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:25.659171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:25.660000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:25.660000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:25.660000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:25.660000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:25.660000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:25.660000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:26.082322 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:26.082674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:26.082861 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:26.084288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:26.084623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.085516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:26.085516 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.085516 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:26.085516 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:26.085516 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.085516 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:26.085516 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.085516 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:26.087055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.087055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:26.087055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:26.087055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:26.087055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:26.087055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:26.087055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:26.087055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.087055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:26.087055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:26.087055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:26.087055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.087433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:26.087433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:26.087433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:26.087433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:26.087433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:26.087433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:26.087433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:26.087433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:26.087433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.087433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:26.087433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:26.087433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.087433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:26.087753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.087753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:26.089113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:26.089559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.090012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:26.090012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.090012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:26.090234 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:26.090453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:26.090575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:26.090688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:30:26.091234 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:26.091480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.092044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:26.092044 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.092044 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:26.092044 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:26.092044 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.092044 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:26.092044 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.092044 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:26.092870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:26.093262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:26.093262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:26.093262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.093262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:26.093262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:26.093262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.093262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:26.093262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.093262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:26.093769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.093769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:26.093769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:26.093769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:26.093769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:26.093769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:26.093769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:26.093769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:30:26.093769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:26.093769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:26.093769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.093769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:26.094076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:26.094421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.094421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:26.094421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.094421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:26.095040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:26.095344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:26.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:26.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:26.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:26.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:26.131965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:26.132414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:26.132663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:26.132956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:26.133812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:26.134233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.135230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:26.135230 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.135230 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:26.135230 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:26.135230 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.135230 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:26.135230 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.135230 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:26.136721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:26.137720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:26.137720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:26.137720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.137720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:26.137720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:26.137720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.137720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:26.137720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.137720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:26.138662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.138662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:26.138662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:26.138662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:26.138662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:26.138662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:26.138662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:26.138662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:26.138662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:26.138662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.138662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:26.139158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:26.139677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.139677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:26.139677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.139677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:26.140706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:26.141532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:26.142180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.142180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:26.142180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.142180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:26.143827 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:26.144051 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:26.144218 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:26.144426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:26.144871 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:26.145221 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:26.145463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:26.146157 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.146157 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:26.146157 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:26.146157 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.146157 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:26.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.146157 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:26.147155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.147155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:26.147155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:26.147155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:26.147155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:26.147155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:26.147155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:26.147155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.147155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:26.147155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:26.147155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:26.147155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.147686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:26.147686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:26.147686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:26.147686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:26.147686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:26.147686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:26.147686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:26.147686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:26.147686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.147686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:26.147686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:26.147686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.147686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:26.148183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.148183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:26.148533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:26.149182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:26.150004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:26.150004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:26.150004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:30.622606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:30.623412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:30.623412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:30.624585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:30.655662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:30.655878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:30.656514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:30.656514 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:30.656514 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:30.656514 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:30.656514 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:30.656514 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:30.656514 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:30.656514 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:30.657257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:30.657257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:30.657257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:30.657257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:30.657257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:30.657257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:30.657257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:30.657257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:30.657257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:30.657257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:30.657257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:30.657257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:30.657625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:30.657625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:30.657625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:30.657625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:30.657625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:30.657625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:30.657625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:30.657625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:30.657625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:30.657625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:30.657625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:30.657625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:30.657625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:30.658008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:30.658462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:30.658951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:30.658951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:30.658951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:30.658951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:30.658951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:30.658951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:31.082863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:31.083242 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:31.083361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:31.084151 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:31.084484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.085234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:31.085234 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.085234 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:31.085234 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:31.085234 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.085234 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:31.085234 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.085234 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:31.086864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.086864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:31.086864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:31.086864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:31.086864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:31.086864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:31.086864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:31.086864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.086864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:31.086864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:31.086864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:31.086864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.087436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:31.087436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:31.087436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:31.087436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:31.087436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:31.087436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:31.087436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:31.087436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:31.087436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.087436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:31.087436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:31.087436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.087436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:31.088003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:31.088858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.089700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:31.089700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:31.089700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.089700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:31.089700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.089700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:31.090542 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:31.090757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:31.090866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:31.090981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:30:31.091413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:31.091686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.092333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:31.092333 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.092333 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:31.092333 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:31.092333 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.092333 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:31.092333 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.092333 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:31.093036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:31.093461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:31.093461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:31.093461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.093461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:31.093461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:31.093461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.093461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:31.093461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.093461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:31.093880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.093880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:31.093880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:31.093880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:31.093880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:31.093880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:31.093880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:31.093880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:30:31.093880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:31.093880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:31.093880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.093880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:31.094199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:31.094592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.094592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:31.094592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.094592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:31.094910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:31.095360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:31.095820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.095820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:31.095820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:31.095820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.095820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:31.095820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.095820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:31.132422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:31.132725 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:31.132833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:31.132971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:31.133486 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:31.133930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.134689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:31.134689 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.134689 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:31.134689 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:31.134689 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.134689 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:31.134689 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.134689 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:31.135512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:31.136148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:31.136148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:31.136148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.136148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:31.136148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:31.136148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.136148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:31.136148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.136148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:31.136564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.136564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:31.136564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:31.136564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:31.136564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:31.136564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:31.136564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:31.136564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:31.136564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:31.136564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.136564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:31.137059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:31.137679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:31.138368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:31.139165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:31.139165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:31.139165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:31.139165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:31.139165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.139165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:31.139165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:31.139165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.139165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:31.139165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.139165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:31.144165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:31.144557 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:31.144810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:31.145139 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:31.145756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:31.146106 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:31.146303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.147184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:31.147184 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.147184 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:31.147184 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:31.147184 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.147184 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:31.147184 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.147184 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:31.148509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.148509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:31.148509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:31.148509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:31.148509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:31.148509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:31.148509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:31.148509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.148509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:31.148509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:31.148509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:31.148509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.149339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:31.149339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:31.149339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:31.149339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:31.149339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:31.149339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:31.149339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:31.149339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:31.149339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.149339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:31.149339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:31.149339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.149339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:31.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:31.150753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:31.151344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:31.151344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:31.151344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:31.151344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:31.151344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:31.151344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:35.623945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:35.623945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:35.623945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:35.625184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:35.643932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=237 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:30:35.660555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:35.660969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:35.662096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:35.662096 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:35.662096 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:35.662096 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:35.662096 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:35.662096 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:35.662096 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:35.662096 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:35.662852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:35.662852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:35.662852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:35.662852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:35.662852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:35.662852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:35.662852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:35.662852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:35.662852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:35.662852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:35.662852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:35.662852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:35.663582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:35.663582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:35.663582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:35.663582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:35.663582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:35.663582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:35.663582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:35.663582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:35.663582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:35.663582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:35.663582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:35.663582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:35.663582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:35.664378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:35.665126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:35.666095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:35.666095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:35.666095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:35.666095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:35.666095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:35.666095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:36.083488 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:36.084136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:36.084350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:36.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:36.085567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.086950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:36.086950 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.086950 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:36.086950 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:36.086950 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.086950 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:36.086950 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.086950 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:36.088310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.088310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:36.088310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:36.088310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:36.088310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:36.088310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:36.088310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:36.088310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.088310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:36.088310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:36.088310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:36.088310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.088613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:36.088613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:36.088613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:36.088613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:36.088613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:36.088613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:36.088613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:36.088613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:36.088613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.088613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:36.088613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:36.088613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.088613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:36.088867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.088867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:36.089878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:36.090227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.090629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:36.090629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.090629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:36.090906 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:36.091096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:36.091248 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:36.091378 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:30:36.091698 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:36.091882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.092377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:36.092377 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.092377 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:36.092377 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:36.092377 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.092377 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:36.092377 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.092377 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:36.093107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:36.093615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:36.093615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:36.093615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.093615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:36.093615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:36.093615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.093615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:36.093615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.093615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:36.094344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.094344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:36.094344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:36.094344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:36.094344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:36.094344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:36.094344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:36.094344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:30:36.094344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:36.094344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:36.094344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.094344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:36.094876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:36.095399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:36.095907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:36.096476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:36.096476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:36.096476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:36.096476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:36.096476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:36.096476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.096476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:36.096476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:36.096476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.096476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:36.096476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.096476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:36.133284 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:36.133810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:36.134297 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:36.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:36.135055 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:36.135549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.136356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:36.136356 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.136356 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:36.136356 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:36.136356 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.136356 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:36.136356 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.136356 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:36.137590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:36.138486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:36.138486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:36.138486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.138486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:36.138486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:36.138486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.138486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:36.138486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.138486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:36.139361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.139361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:36.139361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:36.139361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:36.139361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:36.139361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:36.139361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:36.139361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:36.139361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:36.139361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.139361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:36.139893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:36.140534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:36.141138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:36.142327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:36.142327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:36.142327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:36.142327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:36.142327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.142327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:36.142327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:36.142327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.142327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:36.142327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.142327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:36.144578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:36.144792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:36.144894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:36.145057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:36.145476 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:36.145905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:36.146092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.146796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:36.146796 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.146796 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:36.146796 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:36.146796 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.146796 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:36.146796 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.146796 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:36.147723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.147723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:36.147723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:36.147723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:36.147723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:36.147723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:36.147723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:36.147723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.147723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:36.147723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:36.147723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:36.147723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.148257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:36.148257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:36.148257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:36.148257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:36.148257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:36.148257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:36.148257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:36.148257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:36.148257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.148257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:36.148257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:36.148257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.148257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:36.148811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.148811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:36.149079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:36.149742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:36.150525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:36.150525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:36.150525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:40.623703 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:40.624442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:40.624505 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:40.625758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:40.660096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:40.660513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:40.661417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:40.661417 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:40.661417 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:40.661417 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:40.661417 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:40.661417 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:40.661417 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:40.661417 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:40.662457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:40.662457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:40.662457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:40.662457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:40.662457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:40.662457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:40.662457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:40.662457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:40.662457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:40.662457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:40.662457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:40.662457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:40.663408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:40.663408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:40.663408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:40.663408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:40.663408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:40.663408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:40.663408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:40.663408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:40.663408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:40.663408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:40.663408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:40.663408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:40.663408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:40.664165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:40.665137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:40.666298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:40.666298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:40.666298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:40.666298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:40.666298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:40.666298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:41.084236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:41.084863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:41.085142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:41.086064 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:41.086423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.087450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:41.087450 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.087450 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:41.087450 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:41.087450 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.087450 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:41.087450 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.087450 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:41.089467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.089467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:41.089467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:41.089467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:41.089467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:41.089467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:41.089467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:41.089467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.089467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:41.089467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:41.089467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:41.089467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.090033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:41.090033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:41.090033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:41.090033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:41.090033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:41.090033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:41.090033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:41.090033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:41.090033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.090033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:41.090033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:41.090033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.090033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:41.090597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.090597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:41.091776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:41.092611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.093715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:41.093715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.093715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:41.093715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:41.093715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:41.093715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:41.093715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:30:41.094039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:41.094039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.095050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:41.095050 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.095050 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:41.095050 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:41.095050 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.095050 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:41.095050 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.095050 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:41.096132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:41.096830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:41.096830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:41.096830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.096830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:41.096830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:41.096830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.096830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:41.096830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.096830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:41.097277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.097277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:41.097277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:41.097277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:41.097277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:41.097277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:41.097277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:41.097277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:30:41.097277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:41.097277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:41.097277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.097277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:41.097841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:41.098457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.098457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:41.098457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.098457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:41.098643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:41.099385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:41.100704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.100704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:41.100704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:41.100704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.100704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:41.100704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.100704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:41.133700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:41.134358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:41.134742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:41.134925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:41.135801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:41.136781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.137305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:41.137305 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.137305 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:41.137305 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:41.137305 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.137305 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:41.137305 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.137305 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:41.138728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:41.139281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:41.139281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:41.139281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.139281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:41.139281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:41.139281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.139281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:41.139281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.139281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:41.139884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.139884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:41.139884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:41.139884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:41.139884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:41.139884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:41.139884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:41.139884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:41.139884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:41.139884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.139884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:41.140360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:41.140848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.140848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:41.140848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.140848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:41.141398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:41.142061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:41.142549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.142549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:41.142549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.142549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:41.144799 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:41.145075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:41.145259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:41.145474 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:41.145981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:41.146437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:41.146640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.147490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:41.147490 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.147490 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:41.147490 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:41.147490 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.147490 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:41.147490 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.147490 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:41.148438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.148438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:41.148438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:41.148438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:41.148438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:41.148438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:41.148438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:41.148438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.148438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:41.148438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:41.148438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:41.148438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.148929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:41.148929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:41.148929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:41.148929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:41.148929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:41.148929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:41.148929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:41.148929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:41.148929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.148929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:41.148929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:41.148929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.148929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:41.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:41.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:41.150292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:41.150836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:41.150836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:41.150836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:45.624471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:45.625287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:45.625287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:45.626223 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:45.656502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:45.656880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.657631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:45.657631 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:45.657631 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:45.657631 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:45.657631 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:45.657631 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:45.657631 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.657631 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:45.658375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.658375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:45.658375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:45.658375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:45.658375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:45.658375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:45.658375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:45.658375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:45.658375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:45.658375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:45.658375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:45.658375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:45.658983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:45.658983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:45.658983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:45.658983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:45.658983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:45.658983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:45.658983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:45.658983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:45.658983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:45.658983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:45.658983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:45.658983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:45.658983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:45.659738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:45.660631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:45.661554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:45.661554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:45.661554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:45.661554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:45.661554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.661554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:45.728736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:45.729019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:45.729193 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:45.737843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 07 15:30:45.738068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Number of inconsistencies found at create/update: networks=2, security_groups=1 {{(pid=105503) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 07 15:30:45.738153 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:30:45.771448 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, resource={'id': '8b456c85-5e2d-47f8-9886-ebf65932cf32', 'name': 'private', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4a00c537-ed4f-405c-b241-82e7bc2cf3fa', '9fc89727-343d-403a-acd7-db6b4950e7b1'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:25:31Z', 'updated_at': '2026-05-07T15:25:34Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 42799}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:45.771824 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:45.772431 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-b0f8152d-626b-440e-9383-919ec905ab38, external_ids_update=None, columns={'tag': 42799}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:45.772637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.773486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:45.773486 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:45.773486 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:45.773486 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:45.773486 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:45.773486 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:45.773486 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.773486 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:45.774660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.774660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:45.774660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:30:45.774660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:30:45.774660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:30:45.774660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:30:45.774660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:30:45.774660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:45.774660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:45.774660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:45.774660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:45.775265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:45.775897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:45.775897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.775897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:45.775897 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:30:45.803923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, resource={'id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'name': 'public', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6aa26064-0a62-428b-a8ed-fef52fc73984', 'adf79775-4dfa-4110-bca0-59be9560299c'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-07T15:25:37Z', 'updated_at': '2026-05-07T15:25:45Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:45.804374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:45.804972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-2b8b3cbc-130c-4894-94e1-ab0729ec7077, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:45.805336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.806070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:45.806070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:45.806070 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:45.806070 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:45.806070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:45.806070 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:45.806070 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.806070 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:45.807131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.807131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:45.807131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:30:45.807131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:30:45.807131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:30:45.807131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:30:45.807131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:30:45.807131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:45.807131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:45.807131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:45.807131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:45.807704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:45.808398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:45.808398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:45.808398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:45.808398 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 042f63ba-c916-4545-b726-f33131504ce5 (type: security_groups) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:30:45.858750 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.db.ovn_revision_numbers_db [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Successfully bumped revision number for resource 042f63ba-c916-4545-b726-f33131504ce5 (type: security_groups) to 9
May 07 15:30:45.858950 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_for_inconsistencies finished in 0.130 seconds
May 07 15:30:46.084887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:46.085246 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:46.085496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:46.086454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:46.086750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.087684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:46.087684 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.087684 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:46.087684 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:46.087684 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.087684 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:46.087684 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.087684 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:46.089152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.089152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:46.089152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:46.089152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:46.089152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:46.089152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:46.089152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:46.089152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.089152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:46.089152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:46.089152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:46.089152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.089805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:46.089805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:46.089805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:46.089805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:46.089805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:46.089805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:46.089805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:46.089805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:46.089805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.089805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:46.089805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:46.089805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.089805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:46.090357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.090357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:46.090724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:46.091325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.091802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:46.091802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.091802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:46.092509 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:46.092774 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:46.092948 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:46.093145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:30:46.093750 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:46.094266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.094818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:46.094818 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.094818 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:46.094818 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:46.094818 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.094818 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:46.094818 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.094818 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:46.096188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:46.096641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:46.096641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:46.096641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.096641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:46.096641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:46.096641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.096641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:46.096641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.096641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:46.097264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.097264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:46.097264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:46.097264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:46.097264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:46.097264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:46.097264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:46.097264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:30:46.097264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:46.097264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:46.097264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.097264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:46.097601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:46.097970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.097970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:46.097970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.097970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:46.098219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:46.098701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:46.099164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.099164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:46.099164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:46.099164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.099164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:46.099164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.099164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:46.134265 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:46.134463 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:46.134546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:46.134661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:46.135156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:46.135521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:46.136301 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.136301 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:46.136301 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:46.136301 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.136301 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:46.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.136301 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:46.137542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:46.138168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:46.138168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:46.138168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.138168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:46.138168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:46.138168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.138168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:46.138168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.138168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:46.139072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.139072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:46.139072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:46.139072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:46.139072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:46.139072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:46.139072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:46.139072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:46.139072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:46.139072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.139072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:46.139604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:46.140364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.140364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:46.140364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.140364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:46.140835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:46.141337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:46.141795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.141795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:46.141795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.141795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:46.145115 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:46.145469 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:46.145654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:46.145869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:46.146301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:46.146627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:46.146796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.147494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:46.147494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.147494 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:46.147494 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:46.147494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.147494 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:46.147494 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.147494 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:46.148307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.148307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:46.148307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:46.148307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:46.148307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:46.148307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:46.148307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:46.148307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.148307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:46.148307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:46.148307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:46.148307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.148649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:46.148649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:46.148649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:46.148649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:46.148649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:46.148649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:46.148649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:46.148649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:46.148649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.148649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:46.148649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:46.148649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.148649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:46.148987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.148987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:46.149215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:46.149723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:46.150238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:46.150238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:46.150238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:50.625790 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:50.626530 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:50.626530 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:50.627663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:50.662449 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:50.662725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:50.663371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:50.663371 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:50.663371 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:50.663371 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:50.663371 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:50.663371 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:50.663371 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:50.663371 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:50.664322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:50.664322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:50.664322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:50.664322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:50.664322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:50.664322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:50.664322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:50.664322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:50.664322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:50.664322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:50.664322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:50.664322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:50.664909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:50.664909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:50.664909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:50.664909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:50.664909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:50.664909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:50.664909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:50.664909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:50.664909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:50.664909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:50.664909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:50.664909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:50.664909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:50.665808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:50.666726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:50.667589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:50.667589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:50.667589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:50.667589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:50.667589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:50.667589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:51.085299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:51.085768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:51.085955 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:51.086870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:51.087175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.088040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:51.088040 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.088040 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:51.088040 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:51.088040 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.088040 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:51.088040 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.088040 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:51.089632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.089632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:51.089632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:51.089632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:51.089632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:51.089632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:51.089632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:51.089632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.089632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:51.089632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:51.089632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:51.089632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.090115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:51.090115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:51.090115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:51.090115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:51.090115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:51.090115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:51.090115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:51.090115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:51.090115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.090115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:51.090115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:51.090115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.090115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:51.090608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.090608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:51.091461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:51.092055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.092723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:51.092723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.092723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:51.092868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:51.093023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:51.093157 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:51.093300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:30:51.093754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:51.094036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.094709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:51.094709 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.094709 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:51.094709 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:51.094709 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.094709 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:51.094709 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.094709 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:51.095488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:51.096084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:51.096084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:51.096084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.096084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:51.096084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:51.096084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.096084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:51.096084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.096084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:51.096440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.096440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:51.096440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:51.096440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:51.096440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:51.096440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:51.096440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:51.096440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:30:51.096440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:51.096440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:51.096440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.096440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:51.097612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:51.098185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:51.098674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:51.099351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:51.099351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:51.099351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:51.099351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:51.099351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:51.099351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.099351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:51.099351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:51.099351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.099351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:51.099351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.099351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:51.134872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:51.135334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:51.135515 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:51.135715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:51.136244 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:51.136511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.137245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:51.137245 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.137245 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:51.137245 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:51.137245 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.137245 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:51.137245 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.137245 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:51.138427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:51.139352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:51.139352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:51.139352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.139352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:51.139352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:51.139352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.139352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:51.139352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.139352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:51.139911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.139911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:51.139911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:51.139911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:51.139911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:51.139911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:51.139911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:51.139911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:51.139911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:51.139911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.139911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:51.140796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:51.141659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:51.142279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:51.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:51.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:51.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:51.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:51.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:51.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:51.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:51.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:51.145640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:51.145938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:51.146039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:51.146194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:51.146690 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:51.147068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:51.147284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.147997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:51.147997 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.147997 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:51.147997 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:51.147997 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.147997 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:51.147997 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.147997 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:51.148793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.148793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:51.148793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:51.148793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:51.148793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:51.148793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:51.148793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:51.148793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.148793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:51.148793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:51.148793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:51.148793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.149309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:51.149309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:51.149309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:51.149309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:51.149309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:51.149309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:51.149309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:51.149309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:51.149309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.149309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:51.149309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:51.149309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.149309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:51.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:51.150363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:51.151025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:51.151025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:51.151025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:51.151025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:51.151025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:51.151025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:55.626960 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:55.627706 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:55.627822 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:55.629478 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:55.665694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:55.666698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:55.667594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:55.667594 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:55.667594 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:55.667594 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:55.667594 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:55.667594 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:55.667594 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:55.667594 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:55.668582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:55.668582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:55.668582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:55.668582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:55.668582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:55.668582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:55.668582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:55.668582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:55.668582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:55.668582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:55.668582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:55.668582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:55.669169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:55.669169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:55.669169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:55.669169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:55.669169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:55.669169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:55.669169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:55.669169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:55.669169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:55.669169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:55.669169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:55.669169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:55.669169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:55.669755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:55.670472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:55.671278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:55.671278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:55.671278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:55.671278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:55.671278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:55.671278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:56.085758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:56.086151 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:56.086402 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:56.087904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:56.088270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.089158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:56.089158 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.089158 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:56.089158 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:56.089158 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.089158 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:56.089158 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.089158 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:56.090739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.090739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:56.090739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:56.090739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:56.090739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:56.090739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:56.090739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:56.090739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.090739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:56.090739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:56.090739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:56.090739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.091144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:56.091144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:56.091144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:56.091144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:56.091144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:56.091144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:56.091144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:56.091144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:56.091144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.091144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:56.091144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:56.091144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.091144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:56.091547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.091547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:56.095658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:56.096441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.097258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:56.097258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.097258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:56.097258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:56.097258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:56.097258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:56.097258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:30:56.097258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:56.097647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.098019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:56.098019 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.098019 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:56.098019 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:56.098019 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.098019 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:56.098019 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.098019 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:56.099390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:56.099808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:56.099808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:56.099808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.099808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:56.099808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:56.099808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.099808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:56.099808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.099808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:56.101124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.101124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:56.101124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:56.101124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:56.101124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:56.101124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:56.101124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:56.101124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:30:56.101124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:56.101124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:56.101124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.101124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:56.101513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:56.101905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.101905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:56.101905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.101905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:56.102891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:56.103434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:56.104356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.104356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:56.104356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:56.104356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.104356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:56.104356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.104356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:56.135193 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:56.135619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:56.135870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:56.136142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:56.136768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:56.137257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.138064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:56.138064 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.138064 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:56.138064 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:56.138064 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.138064 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:56.138064 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.138064 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:30:56.139367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:30:56.140017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:56.140017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:30:56.140017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.140017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:30:56.140017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:30:56.140017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.140017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:30:56.140017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.140017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:30:56.140438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.140438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:56.140438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:56.140438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:56.140438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:56.140438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:56.140438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:56.140438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:56.140438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:56.140438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.140438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:56.140929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:56.141527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.141527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:56.141527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.141527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:56.141895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:56.142491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:56.142999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.142999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:56.142999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.142999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:30:56.146004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:30:56.146630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:30:56.146738 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:30:56.146911 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:30:56.147425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:56.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:30:56.147960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.148696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:30:56.148696 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.148696 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:30:56.148696 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:30:56.148696 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.148696 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:30:56.148696 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.148696 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:30:56.149637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.149637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:30:56.149637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:56.149637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:30:56.149637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:30:56.149637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:56.149637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:56.149637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.149637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:56.149637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:56.149637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:30:56.149637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.150005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:30:56.150005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:56.150005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:30:56.150005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:56.150005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:30:56.150005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:30:56.150005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:56.150005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:30:56.150005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.150005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:30:56.150005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:30:56.150005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.150005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:30:56.150420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.150420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:30:56.150656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:30:56.151147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:30:56.152118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:30:56.152118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:30:56.152118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:00.628157 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:00.628831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:00.628831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:00.629411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:00.660253 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:00.660799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:00.661684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:00.661684 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:00.661684 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:00.661684 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:00.661684 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:00.661684 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:00.661684 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:00.661684 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:00.662684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:00.662684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:00.662684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:00.662684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:00.662684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:00.662684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:00.662684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:00.662684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:00.662684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:00.662684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:00.662684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:00.662684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:00.663285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:00.663285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:00.663285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:00.663285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:00.663285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:00.663285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:00.663285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:00.663285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:00.663285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:00.663285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:00.663285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:00.663285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:00.663285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:00.663813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:00.664501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:00.665417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:00.665417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:00.665417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:00.665417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:00.665417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:00.665417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:01.086358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:01.086830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:01.087054 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:01.088103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:01.088468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.089486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:01.089486 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.089486 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:01.089486 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:01.089486 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.089486 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:01.089486 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.089486 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:01.090961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.090961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:01.090961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:01.090961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:01.090961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:01.090961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:01.090961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:01.090961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.090961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:01.090961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:01.090961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:01.090961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.091563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:01.091563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:01.091563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:01.091563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:01.091563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:01.091563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:01.091563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:01.091563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:01.091563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.091563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:01.091563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:01.091563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.091563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:01.092095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.092095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:01.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:01.094138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.094984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:01.094984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.094984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:01.096167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:01.096524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:01.096697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:01.096872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:31:01.097373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:01.097670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.098350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:01.098350 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.098350 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:01.098350 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:01.098350 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.098350 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:01.098350 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.098350 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:01.099152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:01.099876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:01.099876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:01.099876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.099876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:01.099876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:01.099876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.099876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:01.099876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.099876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:01.100286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.100286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:01.100286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:01.100286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:01.100286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:01.100286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:01.100286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:01.100286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:31:01.100286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:01.100286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:01.100286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.100286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:01.100609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:01.100956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:01.101352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:01.101822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:01.101822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:01.101822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:01.101822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:01.101822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:01.101822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.101822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:01.101822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:01.101822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.101822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:01.101822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.101822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:01.135627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:01.135958 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:01.136143 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:01.136390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:01.136870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:01.137246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.138005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:01.138005 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.138005 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:01.138005 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:01.138005 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.138005 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:01.138005 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.138005 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:01.139423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:01.140137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:01.140137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:01.140137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.140137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:01.140137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:01.140137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.140137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:01.140137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.140137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:01.140909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.140909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:01.140909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:01.140909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:01.140909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:01.140909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:01.140909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:01.140909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:01.140909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:01.140909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.140909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:01.141456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:01.142043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.142043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:01.142043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.142043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:01.143894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:01.144525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:01.145033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.145033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:01.145033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.145033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:01.146460 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:01.146698 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:01.146817 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:01.146962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:01.147461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:01.147809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:01.147998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.148715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:01.148715 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.148715 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:01.148715 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:01.148715 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.148715 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:01.148715 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.148715 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:01.149533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.149533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:01.149533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:01.149533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:01.149533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:01.149533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:01.149533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:01.149533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.149533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:01.149533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:01.149533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:01.149533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.149848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:01.149848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:01.149848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:01.149848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:01.149848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:01.149848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:01.149848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:01.149848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:01.149848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.149848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:01.149848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:01.149848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.149848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:01.150161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.150161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:01.150458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:01.150848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:01.151365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:01.151365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:01.151365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:05.628742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:05.629530 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:05.629530 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:05.630373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:05.664016 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:05.664507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:05.665339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:05.665339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:05.665339 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:05.665339 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:05.665339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:05.665339 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:05.665339 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:05.665339 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:05.667048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:05.667048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:05.667048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:05.667048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:05.667048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:05.667048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:05.667048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:05.667048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:05.667048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:05.667048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:05.667048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:05.667048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:05.667693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:05.667693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:05.667693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:05.667693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:05.667693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:05.667693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:05.667693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:05.667693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:05.667693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:05.667693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:05.667693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:05.667693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:05.667693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:05.668261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:05.668261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:05.668813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:05.670597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:05.671930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:05.671930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:05.671930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:06.086987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:06.087660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:06.087875 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:06.088690 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:06.089036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.089870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:06.089870 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.089870 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:06.089870 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:06.089870 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.089870 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:06.089870 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.089870 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:06.091277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.091277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:06.091277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:06.091277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:06.091277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:06.091277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:06.091277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:06.091277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.091277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:06.091277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:06.091277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:06.091277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.091715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:06.091715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:06.091715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:06.091715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:06.091715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:06.091715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:06.091715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:06.091715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:06.091715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.091715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:06.091715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:06.091715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.091715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:06.092002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.092002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:06.092873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:06.093298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.093707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:06.093707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.093707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:06.096660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:06.097072 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:06.097262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:06.097442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:31:06.097957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:06.098280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.098981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:06.098981 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.098981 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:06.098981 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:06.098981 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.098981 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:06.098981 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.098981 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:06.099918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:06.100406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:06.100406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:06.100406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.100406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:06.100406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:06.100406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.100406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:06.100406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.100406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:06.100948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.100948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:06.100948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:06.100948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:06.100948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:06.100948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:06.100948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:06.100948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:31:06.100948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:06.100948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:06.100948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.100948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:06.101819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:06.102672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:06.103236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:06.103962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:06.103962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:06.103962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:06.103962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:06.103962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:06.103962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.103962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:06.103962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:06.103962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.103962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:06.103962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.103962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:06.136088 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:06.136659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:06.136822 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:06.137024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:06.137851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:06.138020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:06.138824 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.138824 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:06.138824 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:06.138824 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.138824 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:06.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:06.139984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:06.140655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:06.140655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:06.140655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.140655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:06.140655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:06.140655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.140655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:06.140655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.140655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:06.141371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.141371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:06.141371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:06.141371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:06.141371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:06.141371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:06.141371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:06.141371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:06.141371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:06.141371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.141371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:06.142498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:06.143737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:06.144306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:06.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:06.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:06.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:06.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:06.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:06.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:06.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:06.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:06.146853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:06.147219 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:06.147400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:06.147559 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:06.147982 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:06.148387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:06.148613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.149270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:06.149270 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.149270 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:06.149270 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:06.149270 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.149270 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:06.149270 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.149270 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:06.150079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.150079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:06.150079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:06.150079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:06.150079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:06.150079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:06.150079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:06.150079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.150079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:06.150079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:06.150079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:06.150079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.150625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:06.150625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:06.150625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:06.150625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:06.150625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:06.150625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:06.150625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:06.150625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:06.150625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.150625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:06.150625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:06.150625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.150625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:06.151170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:06.151598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:06.151906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:06.151906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:06.151906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:10.629383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:10.630123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:10.630123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:10.630790 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:10.663812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:10.664171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:10.665018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:10.665018 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:10.665018 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:10.665018 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:10.665018 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:10.665018 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:10.665018 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:10.665018 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:10.665880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:10.665880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:10.665880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:10.665880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:10.665880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:10.665880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:10.665880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:10.665880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:10.665880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:10.665880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:10.665880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:10.665880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:10.666598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:10.666598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:10.666598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:10.666598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:10.666598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:10.666598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:10.666598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:10.666598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:10.666598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:10.666598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:10.666598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:10.666598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:10.666598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:10.667303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:10.668269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:10.669370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:10.669370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:10.669370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:10.669370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:10.669370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:10.669370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:11.087607 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:11.088089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:11.088227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:11.088963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:11.089286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.090131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:11.090131 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.090131 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:11.090131 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:11.090131 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.090131 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:11.090131 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.090131 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:11.091807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.091807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:11.091807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:11.091807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:11.091807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:11.091807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:11.091807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:11.091807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.091807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:11.091807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:11.091807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:11.091807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.092454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:11.092454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:11.092454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:11.092454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:11.092454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:11.092454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:11.092454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:11.092454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:11.092454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.092454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:11.092454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:11.092454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.092454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:11.093034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.093034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:11.094034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:11.094708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.095239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:11.095239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.095239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:11.097068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:11.097356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:11.097468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:11.097603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:31:11.098105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:11.098424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.099225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:11.099225 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.099225 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:11.099225 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:11.099225 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.099225 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:11.099225 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.099225 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:11.099941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:11.100387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:11.100387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:11.100387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.100387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:11.100387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:11.100387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.100387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:11.100387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.100387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:11.100738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.100738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:11.100738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:11.100738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:11.100738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:11.100738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:11.100738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:11.100738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:31:11.100738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:11.100738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:11.100738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.100738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:11.101083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:11.101524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.101524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:11.101524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.101524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:11.102150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:11.102643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:11.103136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.103136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:11.103136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:11.103136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.103136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:11.103136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.103136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:11.136611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:11.137038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:11.137227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:11.137464 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:11.138106 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:11.138667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.139458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:11.139458 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.139458 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:11.139458 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:11.139458 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.139458 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:11.139458 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.139458 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:11.141074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:11.141771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:11.141771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:11.141771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.141771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:11.141771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:11.141771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.141771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:11.141771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.141771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:11.142638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.142638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:11.142638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:11.142638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:11.142638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:11.142638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:11.142638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:11.142638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:11.142638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:11.142638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.142638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:11.143145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:11.143970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:11.144588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:11.145403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:11.145403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:11.145403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:11.145403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:11.145403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.145403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:11.145403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:11.145403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.145403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:11.145403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.145403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:11.147292 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:11.147562 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:11.147816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:11.148161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:11.148472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:11.148625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.149135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:11.149135 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.149135 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:11.149135 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:11.149135 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.149135 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:11.149135 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.149135 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:11.149986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.149986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:11.149986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:11.149986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:11.149986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:11.149986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:11.149986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:11.149986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.149986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:11.149986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:11.149986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:11.149986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.150364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:11.150364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:11.150364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:11.150364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:11.150364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:11.150364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:11.150364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:11.150364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:11.150364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.150364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:11.150364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:11.150364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.150364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:11.150738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.150738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:11.151086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:11.151578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:11.152092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:11.152092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:11.152092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:15.630092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:15.630919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:15.630919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:15.632147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:15.662984 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:15.663438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:15.664301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:15.664301 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:15.664301 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:15.664301 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:15.664301 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:15.664301 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:15.664301 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:15.664301 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:15.665254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:15.665254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:15.665254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:15.665254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:15.665254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:15.665254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:15.665254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:15.665254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:15.665254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:15.665254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:15.665254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:15.665254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:15.665872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:15.665872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:15.665872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:15.665872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:15.665872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:15.665872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:15.665872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:15.665872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:15.665872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:15.665872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:15.665872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:15.665872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:15.665872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:15.666510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:15.667366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:15.668658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:15.668658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:15.668658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:15.668658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:15.668658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:15.668658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:16.088135 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:16.088697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:16.088886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:16.089731 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:16.090022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.090897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:16.090897 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.090897 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:16.090897 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:16.090897 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.090897 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:16.090897 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.090897 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:16.092309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.092309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:16.092309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:16.092309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:16.092309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:16.092309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:16.092309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:16.092309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.092309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:16.092309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:16.092309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:16.092309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.092890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:16.092890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:16.092890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:16.092890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:16.092890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:16.092890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:16.092890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:16.092890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:16.092890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.092890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:16.092890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:16.092890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.092890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:16.093494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.093494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:16.093935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:16.094618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.095438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:16.095438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.095438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:16.097378 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:16.097701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:16.097816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:16.097937 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:31:16.098390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:16.098725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.099338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:16.099338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.099338 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:16.099338 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:16.099338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.099338 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:16.099338 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.099338 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:16.100086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:16.100629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:16.100629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:16.100629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.100629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:16.100629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:16.100629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.100629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:16.100629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.100629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:16.100938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.100938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:16.100938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:16.100938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:16.100938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:16.100938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:16.100938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:16.100938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:31:16.100938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:16.100938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:16.100938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.100938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:16.101275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:16.101665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.101665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:16.101665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.101665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:16.101989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:16.102425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:16.102900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.102900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:16.102900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:16.102900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.102900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:16.102900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.102900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:16.136945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:16.137365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:16.137539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:16.137732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:16.138226 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:16.138600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.139366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:16.139366 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.139366 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:16.139366 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:16.139366 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.139366 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:16.139366 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.139366 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:16.140505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:16.141112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:16.141112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:16.141112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.141112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:16.141112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:16.141112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.141112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:16.141112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.141112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:16.141842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.141842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:16.141842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:16.141842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:16.141842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:16.141842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:16.141842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:16.141842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:16.141842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:16.141842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.141842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:16.142352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:16.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:16.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.142919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:16.143769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:16.144505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:16.145261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.145261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:16.145261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.145261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:16.147595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:16.147825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:16.147937 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:16.148101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:16.148469 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:16.148789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:16.148957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.149606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:16.149606 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.149606 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:16.149606 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:16.149606 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.149606 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:16.149606 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.149606 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:16.150470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.150470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:16.150470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:16.150470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:16.150470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:16.150470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:16.150470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:16.150470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.150470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:16.150470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:16.150470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:16.150470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.150820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:16.150820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:16.150820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:16.150820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:16.150820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:16.150820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:16.150820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:16.150820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:16.150820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.150820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:16.150820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:16.150820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.150820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:16.151135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.151135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:16.151708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:16.152320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:16.153023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:16.153023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:16.153023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:20.631596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:20.632309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:20.632309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:20.632961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:20.662118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:20.662696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:20.663451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:20.663451 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:20.663451 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:20.663451 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:20.663451 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:20.663451 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:20.663451 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:20.663451 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:20.664231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:20.664231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:20.664231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:20.664231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:20.664231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:20.664231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:20.664231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:20.664231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:20.664231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:20.664231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:20.664231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:20.664231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:20.664944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:20.664944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:20.664944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:20.664944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:20.664944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:20.664944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:20.664944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:20.664944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:20.664944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:20.664944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:20.664944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:20.664944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:20.664944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:20.665538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:20.666185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:20.667019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:20.667019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:20.667019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:20.667019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:20.667019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:20.667019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:21.088860 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:21.089783 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:21.089998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:21.091087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:21.091640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.092673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:21.092673 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.092673 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:21.092673 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:21.092673 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.092673 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:21.092673 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.092673 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:21.093773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.093773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:21.093773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:21.093773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:21.093773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:21.093773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:21.093773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:21.093773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.093773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:21.093773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:21.093773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:21.093773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.094586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:21.094586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:21.094586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:21.094586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:21.094586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:21.094586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:21.094586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:21.094586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:21.094586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.094586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:21.094586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:21.094586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.094586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:21.095258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:21.099930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.100767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:21.100767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:21.100767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.100767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:21.100767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.100767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:21.100767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:21.100767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:21.100767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:21.100767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:21.101247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.101870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:21.102724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.102724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:21.102724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.102724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:21.102724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:21.102724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:21.102724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:21.102724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:21.102724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:21.102724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:31:21.102724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:21.102724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:21.103297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:21.104008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.104652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.105246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:21.137588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:21.138138 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:21.138310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:21.138467 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:21.139235 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:21.139653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.140777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:21.140777 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.140777 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:21.140777 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:21.140777 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.140777 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:21.140777 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.140777 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:21.141906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:21.142639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:21.142639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:21.142639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.142639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:21.142639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:21.142639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.142639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:21.142639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.142639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:21.143143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.143143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:21.143143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:21.143143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:21.143143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:21.143143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:21.143143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:21.143143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:21.143143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:21.143143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.143143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:21.143717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:21.144374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:21.145062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:21.145889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:21.145889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:21.145889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:21.145889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:21.145889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.145889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:21.145889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:21.145889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.145889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:21.145889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.145889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:21.148146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:21.148757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:21.148943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:21.149275 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:21.149773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:21.150125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:21.150555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.152408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:21.152408 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.152408 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:21.152408 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:21.152408 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.152408 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:21.152408 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.152408 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:21.153665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.153665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:21.153665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:21.153665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:21.153665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:21.153665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:21.153665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:21.153665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.153665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:21.153665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:21.153665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:21.153665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:21.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:21.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:21.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:21.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:21.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:21.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:21.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:21.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:21.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:21.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:21.154479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.154479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:21.156312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:21.156854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:21.157407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:21.157407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:21.157407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:25.634596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:25.635427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:25.635615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:25.637852 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:25.683527 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:25.683756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:25.684658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:25.684658 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:25.684658 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:25.684658 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:25.684658 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:25.684658 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:25.684658 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:25.684658 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:25.685525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:25.685525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:25.685525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:25.685525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:25.685525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:25.685525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:25.685525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:25.685525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:25.685525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:25.685525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:25.685525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:25.685525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:25.685921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:25.685921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:25.685921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:25.685921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:25.685921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:25.685921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:25.685921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:25.685921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:25.685921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:25.685921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:25.685921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:25.685921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:25.685921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:25.686314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:25.686314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:25.686583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:25.687063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:25.687622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:25.687622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:25.687622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:26.091861 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:26.092394 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:26.092582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:26.094952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:26.095202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.095917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:26.095917 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.095917 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:26.095917 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:26.095917 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.095917 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:26.095917 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.095917 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:26.097103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.097103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:26.097103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:26.097103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:26.097103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:26.097103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:26.097103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:26.097103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.097103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:26.097103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:26.097103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:26.097103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:26.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:26.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:26.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:26.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:26.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:26.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:26.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:26.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:26.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:26.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:26.097772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.097772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:26.100015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:26.100586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.101053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:26.101053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.101053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:26.101053 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:26.101053 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:26.101053 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:26.101053 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:31:26.101346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:26.101346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.101849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:26.101849 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.101849 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:26.101849 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:26.101849 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.101849 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:26.101849 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.101849 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:26.102663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:26.103080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:26.103080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:26.103080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.103080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:26.103080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:26.103080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.103080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:26.103080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.103080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:26.104175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.104175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:26.104175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:26.104175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:26.104175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:26.104175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:26.104175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:26.104175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:31:26.104175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:26.104175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:26.104175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.104175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:26.104586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:26.104939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:26.105288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:26.105736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:26.105736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:26.105736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:26.105736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:26.105736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:26.105736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.105736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:26.105736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:26.105736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.105736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:26.105736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.105736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:26.138465 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:26.139550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:26.139670 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:26.139825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:26.140551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:26.140813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.141444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:26.141444 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.141444 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:26.141444 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:26.141444 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.141444 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:26.141444 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.141444 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:26.142539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:26.142923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:26.142923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:26.142923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.142923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:26.142923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:26.142923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.142923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:26.142923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.142923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:26.143468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.143468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:26.143468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:26.143468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:26.143468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:26.143468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:26.143468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:26.143468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:26.143468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:26.143468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.143468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:26.143810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:26.144142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.144142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:26.144142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.144142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:26.144670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:26.145600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:26.146091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.146091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:26.146091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.146091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:26.148554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:26.148796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:26.148952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:26.149116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:26.149461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:26.149639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:26.150103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.150325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:26.150325 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.150325 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:26.150325 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:26.150325 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.150325 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:26.150325 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.150325 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:26.151307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.151307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:26.151307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:26.151307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:26.151307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:26.151307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:26.151307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:26.151307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.151307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:26.151307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:26.151307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:26.151307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.151625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:26.151625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:26.151625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:26.151625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:26.151625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:26.151625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:26.151625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:26.151625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:26.151625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.151625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:26.151625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:26.151625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.151625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:26.151926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.151926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:26.152476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:26.152859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:26.153370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:26.153370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:26.153370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:29.671130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=e6b71ef5-4cc6-44a4-8bba-ecd2a64c099f, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'neutron:subnet_ids': '6aa26064-0a62-428b-a8ed-fef52fc73984 adf79775-4dfa-4110-bca0-59be9560299c'}, gateway_chassis=[], ha_chassis_group=f0498155-c1ad-4537-a246-90b793fb299c, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:f6:db:65', name='lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c', networks=['172.24.5.213/24', '2001:db8::15a/64'], options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=105503) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 07 15:31:30.635324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:30.636026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:30.636227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:30.637512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:30.671718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:30.672532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:30.672776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:30.672776 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:30.672776 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:30.672776 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:30.672776 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:30.672776 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:30.672776 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:30.672776 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:30.673473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:30.673473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:30.673473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:30.673473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:30.673473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:30.673473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:30.673473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:30.673473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:30.673473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:30.673473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:30.673473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:30.673473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:30.674197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:30.674197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:30.674197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:30.674197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:30.674197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:30.674197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:30.674197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:30.674197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:30.674197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:30.674197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:30.674197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:30.674197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:30.674197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:30.674810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:30.675615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:30.676452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:30.676452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:30.676452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:30.676452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:30.676452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:30.676452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:31.092537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:31.093160 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:31.093386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:31.094174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:31.094437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:31.094606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.095262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:31.095262 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.095262 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:31.095262 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:31.095262 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.095262 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:31.095262 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.095262 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:31.096984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.096984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:31.096984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:31.096984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:31.096984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:31.096984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:31.096984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:31.096984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.096984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:31.096984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:31.096984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:31.096984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.097742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:31.097742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:31.097742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:31.097742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:31.097742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:31.097742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:31.097742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:31.097742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:31.097742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.097742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:31.097742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:31.097742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.097742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:31.098398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.098398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:31.098689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:31.099329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.099850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:31.099850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.099850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:31.100611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:31.100980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:31.101099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:31.101230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:31:31.101707 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:31.102238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.102981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:31.102981 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.102981 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:31.102981 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:31.102981 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.102981 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:31.102981 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.102981 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:31.103734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:31.104185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:31.104185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:31.104185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.104185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:31.104185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:31.104185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.104185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:31.104185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.104185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:31.104791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.104791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:31.104791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:31.104791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:31.104791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:31.104791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:31.104791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:31.104791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:31:31.104791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:31.104791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:31.104791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.104791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:31.105278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:31.105652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:31.106090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:31.106677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:31.106677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:31.106677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:31.106677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:31.106677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:31.106677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.106677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:31.106677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:31.106677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.106677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:31.106677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.106677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:31.139703 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:31.140199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:31.140329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:31.140483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:31.141233 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:31.141812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:31.142976 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.142976 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:31.142976 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:31.142976 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.142976 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:31.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.142976 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:31.143867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:31.144576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:31.144576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:31.144576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.144576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:31.144576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:31.144576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.144576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:31.144576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.144576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:31.145042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.145042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:31.145042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:31.145042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:31.145042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:31.145042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:31.145042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:31.145042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:31.145042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:31.145042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.145042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:31.145604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:31.146243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:31.146926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:31.147752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:31.147752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:31.147752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:31.147752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:31.147752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.147752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:31.147752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:31.147752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.147752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:31.147752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.147752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:31.148998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:31.149373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:31.149475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:31.149645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:31.150111 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:31.150503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:31.150777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.151473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:31.151473 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.151473 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:31.151473 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:31.151473 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.151473 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:31.151473 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.151473 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:31.152317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.152317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:31.152317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:31.152317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:31.152317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:31.152317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:31.152317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:31.152317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.152317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:31.152317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:31.152317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:31.152317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.152695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:31.152695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:31.152695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:31.152695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:31.152695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:31.152695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:31.152695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:31.152695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:31.152695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.152695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:31.152695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:31.152695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.152695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:31.153046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.153046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:31.153340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:31.153792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:31.154416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:31.154416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:31.154416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:32.837644 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=9ba895e2-6e6e-419e-a379-725fbcdb7ae4, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'neutron:subnet_ids': '2e6720c7-3159-41b9-80a2-fa6cf6a18808'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:6f:d8:b6', name='lrp-0c9e41ba-01dc-483e-a917-8c2bfc366ddb', networks=['192.168.24.1/24'], options={}, peer=[], status={}) old= {{(pid=105503) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 07 15:31:32.867422 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698 synchronized; highest priority chassis 6c608f94-302c-45c4-8e1f-fa23d1659434
May 07 15:31:32.868083 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '4fb571ad-ed2f-4edf-a199-cb52ff142698', 'neutron:router_id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f'}}) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:32.868652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, chassis=6c608f94-302c-45c4-8e1f-fa23d1659434, priority=1, columns={}) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:32.868982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event Traceback (most recent call last):
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2416, in link_network_ha_chassis_group
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     next(self.gen)
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:32.871251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     next(self.gen)
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     with self.create_transaction(
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     self.result = self.commit()
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     raise result.ex
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:32.871854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.event 
May 07 15:31:35.636187 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:35.636850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:35.636920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:35.638122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:35.670218 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:35.670839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:35.671933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:35.671933 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:35.671933 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:35.671933 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:35.671933 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:35.671933 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:35.671933 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:35.671933 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:35.672900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:35.672900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:35.672900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:35.672900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:35.672900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:35.672900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:35.672900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:35.672900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:35.672900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:35.672900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:35.672900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:35.672900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:35.673279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:35.673279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:35.673279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:35.673279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:35.673279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:35.673279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:35.673279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:35.673279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:35.673279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:35.673279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:35.673279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:35.673279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:35.673279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:35.673599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:35.673599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:35.673911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:35.674367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:35.674846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:35.674846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:35.674846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:36.093322 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:36.093753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:36.093934 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:36.094992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:36.095384 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:36.095622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.096514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:36.096514 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.096514 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:36.096514 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:36.096514 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.096514 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:36.096514 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.096514 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:36.098081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.098081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:36.098081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:36.098081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:36.098081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:36.098081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:36.098081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:36.098081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.098081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:36.098081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:36.098081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:36.098081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.098453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:36.098453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:36.098453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:36.098453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:36.098453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:36.098453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:36.098453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:36.098453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:36.098453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.098453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:36.098453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:36.098453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.098453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:36.098791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.098791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:36.099843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:36.100312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.100830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:36.100830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.100830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:36.101156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:36.101461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:36.101635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:36.101820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:31:36.102286 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:36.102487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.103060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:36.103060 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.103060 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:36.103060 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:36.103060 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.103060 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:36.103060 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.103060 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:36.103895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:36.104374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:36.104374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:36.104374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.104374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:36.104374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:36.104374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.104374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:36.104374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.104374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:36.104766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.104766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:36.104766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:36.104766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:36.104766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:36.104766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:36.104766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:36.104766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:31:36.104766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:36.104766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:36.104766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.104766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:36.105115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:36.105501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.105501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:36.105501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.105501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:36.105783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:36.106343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:36.106811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.106811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:36.106811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:36.106811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.106811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:36.106811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.106811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:36.140404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:36.140721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:36.140882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:36.141066 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:36.141579 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:36.141919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.142652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:36.142652 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.142652 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:36.142652 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:36.142652 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.142652 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:36.142652 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.142652 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:36.143828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:36.144487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:36.144487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:36.144487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.144487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:36.144487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:36.144487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.144487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:36.144487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.144487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:36.145225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.145225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:36.145225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:36.145225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:36.145225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:36.145225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:36.145225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:36.145225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:36.145225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:36.145225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.145225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:36.145722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:36.146314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.146314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:36.146314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.146314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:36.147014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:36.147775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:36.148585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.148585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:36.148585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.148585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:36.149562 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:36.149854 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:36.149959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:36.150101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:36.150512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:36.150846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:36.151059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.151779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:36.151779 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.151779 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:36.151779 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:36.151779 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.151779 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:36.151779 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.151779 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:36.152572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.152572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:36.152572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:36.152572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:36.152572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:36.152572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:36.152572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:36.152572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.152572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:36.152572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:36.152572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:36.152572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.152887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:36.152887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:36.152887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:36.152887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:36.152887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:36.152887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:36.152887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:36.152887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:36.152887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.152887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:36.152887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:36.152887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.152887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:36.153261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.153261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:36.153488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:36.153900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:36.154402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:36.154402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:36.154402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:40.636662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:40.637374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:40.637467 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:40.638689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:40.640391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=334,neutron:UPDATE=1 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:31:40.678723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:40.679050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:40.679844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:40.679844 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:40.679844 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:40.679844 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:40.679844 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:40.679844 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:40.679844 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:40.679844 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:40.680754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:40.680754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:40.680754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:40.680754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:40.680754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:40.680754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:40.680754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:40.680754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:40.680754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:40.680754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:40.680754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:40.680754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:40.681350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:40.681350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:40.681350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:40.681350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:40.681350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:40.681350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:40.681350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:40.681350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:40.681350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:40.681350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:40.681350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:40.681350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:40.681350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:40.681972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:40.682737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:40.683525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:40.683525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:40.683525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:40.683525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:40.683525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:40.683525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:41.093714 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:41.094694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:41.094694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:41.095361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:41.095566 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:41.095857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.097938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:41.097938 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.097938 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:41.097938 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:41.097938 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.097938 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:41.097938 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.097938 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:41.099050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.099050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:41.099050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:41.099050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:41.099050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:41.099050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:41.099050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:41.099050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.099050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:41.099050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:41.099050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:41.099050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.100506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:41.100506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:41.100506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:41.100506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:41.100506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:41.100506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:41.100506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:41.100506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:41.100506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.100506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:41.100506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:41.100506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.100506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:41.101284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:41.101755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.102466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:41.102466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:41.102466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.102466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:41.102466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.102466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:41.102466 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:41.104003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:41.104097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:41.104301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:31:41.104760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:41.105064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.105750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:41.105750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.105750 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:41.105750 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:41.105750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.105750 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:41.105750 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.105750 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:41.106282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:41.106749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:41.106749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:41.106749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.106749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:41.106749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:41.106749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.106749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:41.106749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.106749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:41.107014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.107014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:41.107014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:41.107014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:41.107014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:41.107014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:41.107014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:41.107014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:31:41.107014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:41.107014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:41.107014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.107014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:41.107388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:41.108047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:41.108536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:41.109096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:41.109096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:41.109096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:41.109096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:41.109096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:41.109096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.109096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:41.109096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:41.109096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.109096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:41.109096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.109096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:41.140753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:41.141055 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:41.141160 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:41.141316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:41.141912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:41.142439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.144157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:41.144157 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.144157 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:41.144157 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:41.144157 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.144157 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:41.144157 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.144157 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:41.145128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:41.145868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:41.145868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:41.145868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.145868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:41.145868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:41.145868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.145868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:41.145868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.145868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:41.146253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.146253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:41.146253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:41.146253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:41.146253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:41.146253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:41.146253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:41.146253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:41.146253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:41.146253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.146253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:41.147135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:41.147798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:41.148477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:41.150120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:41.150120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:41.150120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:41.150120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:41.150120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.150120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:41.150120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:41.150120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.150120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:41.150120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.150120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:41.150120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:41.150486 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:41.150486 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:41.150486 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:41.150868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:41.151038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:41.151175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.152161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:41.152161 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.152161 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:41.152161 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:41.152161 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.152161 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:41.152161 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.152161 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:41.153154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.153154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:41.153154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:41.153154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:41.153154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:41.153154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:41.153154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:41.153154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.153154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:41.153154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:41.153154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:41.153154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.153584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:41.153584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:41.153584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:41.153584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:41.153584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:41.153584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:41.153584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:41.153584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:41.153584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.153584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:41.153584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:41.153584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.153584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:41.153954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:41.154422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:41.155001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:41.155001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:41.155001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:41.155001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:41.155001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:41.155001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:45.637382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:45.637760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:45.638103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:45.639098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:45.670965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:45.671366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:45.672503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:45.672503 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:45.672503 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:45.672503 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:45.672503 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:45.672503 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:45.672503 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:45.672503 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:45.673426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:45.673426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:45.673426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:45.673426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:45.673426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:45.673426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:45.673426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:45.673426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:45.673426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:45.673426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:45.673426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:45.673426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:45.674399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:45.674399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:45.674399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:45.674399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:45.674399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:45.674399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:45.674399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:45.674399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:45.674399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:45.674399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:45.674399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:45.674399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:45.674399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:45.675192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:45.676069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:45.678037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:45.678037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:45.678037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:45.678037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:45.678037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:45.678037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:46.094202 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:46.094695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:46.094825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:46.095613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:46.095613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:46.095742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.096578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:46.096578 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.096578 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:46.096578 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:46.096578 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.096578 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:46.096578 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.096578 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:46.097648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.097648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:46.097648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:46.097648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:46.097648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:46.097648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:46.097648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:46.097648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.097648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:46.097648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:46.097648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:46.097648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.098023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:46.098023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:46.098023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:46.098023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:46.098023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:46.098023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:46.098023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:46.098023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:46.098023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.098023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:46.098023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:46.098023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.098023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:46.098413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.098413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:46.098770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:46.099242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.099779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:46.099779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.099779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:46.104369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:46.104369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:46.104559 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:46.104598 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:31:46.104887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:46.105342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.105497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:46.105497 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.105497 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:46.105497 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:46.105497 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.105497 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:46.105497 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.105497 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:46.106166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:46.106807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:46.106807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:46.106807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.106807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:46.106807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:46.106807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.106807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:46.106807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.106807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:46.107061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.107061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:46.107061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:46.107061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:46.107061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:46.107061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:46.107061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:46.107061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:31:46.107061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:46.107061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:46.107061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.107061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:46.107436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:46.107792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.107792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:46.107792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.107792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:46.108793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:46.110065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:46.111444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.111444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:46.111444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:46.111444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.111444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:46.111444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.111444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:46.141267 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:46.141782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:46.141937 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:46.142142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:46.142588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:46.142860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.143441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:46.143441 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.143441 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:46.143441 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:46.143441 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.143441 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:46.143441 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.143441 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:46.144282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:46.144667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:46.144667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:46.144667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.144667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:46.144667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:46.144667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.144667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:46.144667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.144667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:46.145170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.145170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:46.145170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:46.145170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:46.145170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:46.145170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:46.145170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:46.145170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:46.145170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:46.145170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.145170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:46.145942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:46.147023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:46.147709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:46.148525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:46.148525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:46.148525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:46.148525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:46.148525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.148525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:46.148525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:46.148525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.148525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:46.148525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.148525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:46.150851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:46.150851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:46.151036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:46.151184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:46.151621 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:46.151966 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:46.152220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.154242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:46.154242 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.154242 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:46.154242 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:46.154242 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.154242 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:46.154242 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.154242 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:46.155492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.155492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:46.155492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:46.155492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:46.155492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:46.155492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:46.155492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:46.155492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.155492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:46.155492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:46.155492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:46.155492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.156499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:46.156499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:46.156499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:46.156499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:46.156499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:46.156499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:46.156499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:46.156499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:46.156499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.156499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:46.156499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:46.156499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.156499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:46.157573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:46.158264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:46.160265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:46.160265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:46.160265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:46.160265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:46.160265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:46.160265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:50.638076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:50.638835 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:50.638835 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:50.639391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:50.704146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:50.704624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:50.705562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:50.705562 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:50.705562 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:50.705562 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:50.705562 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:50.705562 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:50.705562 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:50.705562 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:50.707023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.067 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:50.707023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:50.707023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:50.707023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:50.707023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:50.707023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:50.707023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:50.707023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:50.707023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:50.707023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:50.707023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:50.707023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:50.707410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:50.707410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:50.707410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:50.707410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:50.707410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:50.707410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:50.707410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:50.707410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:50.707410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:50.707410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:50.707410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:50.707410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:50.707410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:50.707924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:50.707924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:50.708809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:50.710255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:50.711251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:50.711251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:50.711251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:51.094820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:51.095339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:51.095468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:51.096089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:51.096297 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:51.096439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.097225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:51.097225 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.097225 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:51.097225 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:51.097225 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.097225 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:51.097225 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.097225 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:51.099513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.099513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:51.099513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:51.099513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:51.099513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:51.099513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:51.099513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:51.099513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.099513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:51.099513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:51.099513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:51.099513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.099879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:51.099879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:51.099879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:51.099879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:51.099879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:51.099879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:51.099879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:51.099879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:51.099879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.099879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:51.099879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:51.099879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.099879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:51.100225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.100225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:51.102127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:51.102591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.103145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:51.103145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.103145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:51.104473 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:51.104757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:51.104859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:51.104978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:31:51.105281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:51.105451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.105823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:51.105823 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.105823 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:51.105823 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:51.105823 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.105823 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:51.105823 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.105823 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:51.106575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:51.106994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:51.106994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:51.106994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.106994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:51.106994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:51.106994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.106994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:51.106994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.106994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:51.107247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.107247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:51.107247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:51.107247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:51.107247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:51.107247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:51.107247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:51.107247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:31:51.107247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:51.107247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:51.107247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.107247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:51.107584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:51.108089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:51.108477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:51.108980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:51.108980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:51.108980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:51.108980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:51.108980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:51.108980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.108980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:51.108980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:51.108980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.108980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:51.108980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.108980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:51.141702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:51.142121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:51.142249 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:51.142412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:51.142752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:51.142982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.143454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:51.143454 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.143454 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:51.143454 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:51.143454 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.143454 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:51.143454 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.143454 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:51.144174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:51.144848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:51.144848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:51.144848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.144848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:51.144848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:51.144848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.144848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:51.144848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.144848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:51.147242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.147242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:51.147242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:51.147242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:51.147242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:51.147242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:51.147242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:51.147242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:51.147242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:51.147242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.147242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:51.147815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:51.148430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:51.149104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:51.152091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:51.152091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:51.152091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:51.152091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:51.152091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.152091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:51.152091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:51.152091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.152091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:51.152091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.152091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:51.152091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:51.153935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:51.154018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:51.154153 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:51.154482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:51.154684 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:51.154828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.155291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:51.155291 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.155291 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:51.155291 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:51.155291 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.155291 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:51.155291 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.155291 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:51.156020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.156020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:51.156020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:51.156020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:51.156020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:51.156020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:51.156020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:51.156020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.156020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:51.156020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:51.156020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:51.156020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.156284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:51.156284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:51.156284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:51.156284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:51.156284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:51.156284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:51.156284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:51.156284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:51.156284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.156284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:51.156284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:51.156284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.156284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:51.156544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.156544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:51.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:51.157212 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:51.157590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:51.157590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:51.157590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:55.638654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:55.639971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:55.640162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:55.641415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:55.671019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:55.671313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:55.672664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:55.672664 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:55.672664 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:55.672664 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:55.672664 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:55.672664 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:55.672664 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:55.672664 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:55.674784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:55.674784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:55.674784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:55.674784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:55.674784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:55.674784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:55.674784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:55.674784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:55.674784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:55.674784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:55.674784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:55.674784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:55.675442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:55.675442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:55.675442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:55.675442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:55.675442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:55.675442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:55.675442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:55.675442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:55.675442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:55.675442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:55.675442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:55.675442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:55.675442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:55.676089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:55.676829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:55.677839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:55.677839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:55.677839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:55.677839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:55.677839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:55.677839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:56.095434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:56.096110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:56.096320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:56.097497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:56.097807 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:56.098036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.098876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:56.098876 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.098876 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:56.098876 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:56.098876 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.098876 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:56.098876 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.098876 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:56.100380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.100380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:56.100380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:56.100380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:56.100380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:56.100380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:56.100380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:56.100380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.100380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:56.100380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:56.100380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:56.100380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.100896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:56.100896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:56.100896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:56.100896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:56.100896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:56.100896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:56.100896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:56.100896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:56.100896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.100896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:56.100896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:56.100896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.100896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:56.101406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.101406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:56.101954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:56.102607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.103372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:56.103372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.103372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:56.104921 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:56.105223 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:56.105340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:56.105465 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:31:56.105892 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:56.106172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.106807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:56.106807 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.106807 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:56.106807 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:56.106807 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.106807 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:56.106807 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.106807 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:56.107593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:56.108021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:56.108021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:56.108021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.108021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:56.108021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:56.108021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.108021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:56.108021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.108021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:56.108457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.108457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:56.108457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:56.108457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:56.108457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:56.108457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:56.108457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:56.108457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:31:56.108457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:56.108457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:56.108457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.108457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:56.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:56.109155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.109155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:56.109155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.109155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:56.109497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:56.109926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:56.110468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.110468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:56.110468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:56.110468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.110468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:56.110468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.110468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:56.142232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:56.142688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:56.142827 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:56.143019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:56.143564 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:56.143891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.144587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:56.144587 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.144587 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:56.144587 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:56.144587 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.144587 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:56.144587 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.144587 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:31:56.145810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:31:56.146517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:56.146517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:31:56.146517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.146517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:31:56.146517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:31:56.146517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.146517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:31:56.146517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.146517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:31:56.147176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.147176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:56.147176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:56.147176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:56.147176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:56.147176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:56.147176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:56.147176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:56.147176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:56.147176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.147176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:56.147701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:56.148297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.148297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:56.148297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.148297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:56.148867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:56.149754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:56.150587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.150587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:56.150587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.150587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:31:56.154031 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:31:56.154328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:31:56.154435 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:31:56.154611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:31:56.155000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:56.155366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:31:56.155573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.156196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:31:56.156196 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.156196 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:31:56.156196 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:31:56.156196 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.156196 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:31:56.156196 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.156196 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:31:56.157098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.157098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:31:56.157098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:56.157098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:31:56.157098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:31:56.157098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:56.157098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:56.157098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.157098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:56.157098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:56.157098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:31:56.157098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.157758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:31:56.157758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:56.157758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:31:56.157758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:56.157758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:31:56.157758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:31:56.157758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:56.157758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:31:56.157758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.157758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:31:56.157758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:31:56.157758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.157758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:31:56.158363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:31:56.159070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:31:56.159957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:31:56.159957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:31:56.159957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:31:56.159957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:31:56.159957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:31:56.159957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:00.639928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:00.640590 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:00.640677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:00.641793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:00.682568 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:00.682946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:00.683841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:00.683841 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:00.683841 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:00.683841 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:00.683841 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:00.683841 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:00.683841 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:00.683841 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:00.684901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:00.684901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:00.684901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:00.684901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:00.684901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:00.684901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:00.684901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:00.684901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:00.684901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:00.684901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:00.684901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:00.684901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:00.685470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:00.685470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:00.685470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:00.685470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:00.685470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:00.685470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:00.685470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:00.685470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:00.685470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:00.685470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:00.685470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:00.685470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:00.685470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:00.686155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:00.687007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:00.687865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:00.687865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:00.687865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:00.687865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:00.687865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:00.687865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:01.095954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:01.096600 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:01.096813 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:01.098351 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:01.098679 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:01.098993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.099868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:01.099868 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.099868 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:01.099868 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:01.099868 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.099868 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:01.099868 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.099868 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:01.101381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.101381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:01.101381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:01.101381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:01.101381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:01.101381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:01.101381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:01.101381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.101381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:01.101381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:01.101381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:01.101381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.101914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:01.101914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:01.101914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:01.101914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:01.101914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:01.101914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:01.101914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:01.101914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:01.101914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.101914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:01.101914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:01.101914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.101914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:01.102293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.102293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:01.103000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:01.103470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.103976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:01.103976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.103976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:01.105361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:01.105697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:01.105868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:01.105991 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:32:01.106382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:01.106589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.107130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:01.107130 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.107130 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:01.107130 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:01.107130 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.107130 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:01.107130 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.107130 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:01.107844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:01.108292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:01.108292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:01.108292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.108292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:01.108292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:01.108292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.108292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:01.108292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.108292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:01.108629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.108629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:01.108629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:01.108629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:01.108629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:01.108629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:01.108629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:01.108629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:32:01.108629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:01.108629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:01.108629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.108629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:01.108973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:01.109371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.109371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:01.109371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.109371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:01.109626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:01.110052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:01.110528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.110528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:01.110528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:01.110528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.110528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:01.110528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.110528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:01.142812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:01.143488 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:01.143643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:01.143806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:01.144368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:01.144658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.145220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:01.145220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.145220 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:01.145220 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:01.145220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.145220 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:01.145220 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.145220 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:01.146217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:01.146639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:01.146639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:01.146639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.146639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:01.146639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:01.146639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.146639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:01.146639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.146639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:01.147166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.147166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:01.147166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:01.147166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:01.147166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:01.147166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:01.147166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:01.147166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:01.147166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:01.147166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.147166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:01.147619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:01.147984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.147984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:01.147984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.147984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:01.148484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:01.148948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:01.149496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.149496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:01.149496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.149496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:01.155231 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:01.155231 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:01.155231 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:01.155231 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:01.155541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:01.155892 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:01.156140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:01.156900 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.156900 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:01.156900 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:01.156900 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.156900 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:01.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.156900 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:01.158119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.158119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:01.158119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:01.158119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:01.158119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:01.158119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:01.158119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:01.158119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.158119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:01.158119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:01.158119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:01.158119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.158561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:01.158561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:01.158561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:01.158561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:01.158561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:01.158561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:01.158561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:01.158561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:01.158561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.158561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:01.158561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:01.158561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.158561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:01.158921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.158921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:01.159549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:01.160009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:01.160555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:01.160555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:01.160555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:05.640583 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:05.641829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:05.642052 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:05.643190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:05.676521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:05.676922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:05.677782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:05.677782 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:05.677782 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:05.677782 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:05.677782 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:05.677782 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:05.677782 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:05.677782 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:05.678530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:05.678530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:05.678530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:05.678530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:05.678530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:05.678530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:05.678530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:05.678530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:05.678530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:05.678530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:05.678530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:05.678530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:05.679120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:05.679120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:05.679120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:05.679120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:05.679120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:05.679120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:05.679120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:05.679120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:05.679120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:05.679120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:05.679120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:05.679120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:05.679120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:05.679769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:05.680494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:05.681374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:05.681374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:05.681374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:05.681374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:05.681374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:05.681374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:06.096629 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:06.097237 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:06.097452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:06.098540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:06.098866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:06.099141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.099968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:06.099968 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.099968 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:06.099968 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:06.099968 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.099968 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:06.099968 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.099968 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:06.101455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.101455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:06.101455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:06.101455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:06.101455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:06.101455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:06.101455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:06.101455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.101455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:06.101455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:06.101455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:06.101455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.102021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:06.102021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:06.102021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:06.102021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:06.102021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:06.102021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:06.102021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:06.102021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:06.102021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.102021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:06.102021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:06.102021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.102021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:06.102596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.102596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:06.103068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:06.103784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.104597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:06.104597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.104597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:06.105798 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:06.106110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:06.106214 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:06.106345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:32:06.106773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:06.107067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.107723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:06.107723 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.107723 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:06.107723 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:06.107723 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.107723 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:06.107723 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.107723 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:06.108628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:06.109060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:06.109060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:06.109060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.109060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:06.109060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:06.109060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.109060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:06.109060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.109060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:06.109422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.109422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:06.109422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:06.109422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:06.109422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:06.109422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:06.109422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:06.109422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:32:06.109422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:06.109422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:06.109422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.109422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:06.109768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:06.110135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.110135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:06.110135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.110135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:06.110382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:06.110815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:06.111302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.111302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:06.111302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:06.111302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.111302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:06.111302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.111302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:06.143501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:06.143856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:06.144025 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:06.144247 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:06.144893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:06.145353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.146153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:06.146153 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.146153 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:06.146153 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:06.146153 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.146153 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:06.146153 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.146153 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:06.147355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:06.147990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:06.147990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:06.147990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.147990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:06.147990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:06.147990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.147990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:06.147990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.147990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:06.148689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.148689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:06.148689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:06.148689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:06.148689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:06.148689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:06.148689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:06.148689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:06.148689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:06.148689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.148689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:06.149234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:06.149853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.149853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:06.149853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.149853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:06.150308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:06.151065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:06.151866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.151866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:06.151866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.151866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:06.154904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:06.155361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:06.155537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:06.155791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:06.156313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:06.156675 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:06.156866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.157549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:06.157549 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.157549 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:06.157549 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:06.157549 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.157549 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:06.157549 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.157549 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:06.158335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.158335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:06.158335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:06.158335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:06.158335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:06.158335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:06.158335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:06.158335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.158335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:06.158335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:06.158335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:06.158335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.158743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:06.158743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:06.158743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:06.158743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:06.158743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:06.158743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:06.158743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:06.158743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:06.158743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.158743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:06.158743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:06.158743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.158743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:06.159089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.159089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:06.159281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:06.159729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:06.160255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:06.160255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:06.160255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:10.641931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:10.642891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:10.642891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:10.644092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:10.673717 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:10.674124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:10.674937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:10.674937 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:10.674937 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:10.674937 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:10.674937 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:10.674937 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:10.674937 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:10.674937 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:10.675854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:10.675854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:10.675854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:10.675854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:10.675854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:10.675854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:10.675854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:10.675854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:10.675854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:10.675854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:10.675854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:10.675854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:10.676761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:10.676761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:10.676761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:10.676761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:10.676761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:10.676761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:10.676761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:10.676761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:10.676761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:10.676761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:10.676761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:10.676761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:10.676761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:10.677320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:10.678169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:10.679653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:10.679653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:10.679653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:10.679653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:10.679653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:10.679653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:11.097339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:11.099810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:11.100001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:11.126645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:11.126645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:11.126645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.126645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:11.126645 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.126645 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:11.126645 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:11.126645 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.126645 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:11.126645 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.126645 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:11.126645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:11.127284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:11.128117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:11.128117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:11.128117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:11.128117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:11.128117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:11.128117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.128117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:11.128117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:11.128117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.128117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:11.128117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.128117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:11.129312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:11.130121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.131828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:11.131828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.131828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:11.131828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:11.131828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:11.131828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:11.131828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:32:11.131828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:11.132198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.132877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:11.133702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.133702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:11.133702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:11.133702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:11.133702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:11.133702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:11.133702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:11.133702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:32:11.133702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:11.133702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:11.133702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.133702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:11.134371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:11.135045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:11.135484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:11.135938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:11.135938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:11.135938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:11.135938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:11.135938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:11.135938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.135938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:11.135938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:11.135938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.135938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:11.135938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.135938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:11.144625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:11.144964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:11.145111 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:11.145293 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:11.145705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:11.145978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.146557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:11.146557 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.146557 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:11.146557 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:11.146557 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.146557 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:11.146557 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.146557 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:11.147412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:11.147794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:11.147794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:11.147794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.147794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:11.147794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:11.147794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.147794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:11.147794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.147794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:11.148306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.148306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:11.148306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:11.148306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:11.148306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:11.148306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:11.148306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:11.148306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:11.148306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:11.148306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.148306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:11.148664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:11.149101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.149101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:11.149101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.149101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:11.149567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:11.150009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:11.150509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.150509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:11.150509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.150509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:11.155314 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:11.155535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:11.155653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:11.155825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:11.156356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:11.156733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:11.156992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.157724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:11.157724 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.157724 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:11.157724 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:11.157724 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.157724 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:11.157724 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.157724 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:11.158616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.158616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:11.158616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:11.158616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:11.158616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:11.158616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:11.158616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:11.158616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.158616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:11.158616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:11.158616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:11.158616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.158999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:11.158999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:11.158999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:11.158999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:11.158999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:11.158999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:11.158999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:11.158999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:11.158999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.158999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:11.158999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:11.158999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.158999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:11.159413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.159413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:11.159641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:11.160086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:11.160895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:11.160895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:11.160895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:15.642640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:15.643403 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:15.643403 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:15.644634 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:15.679232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:15.679639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:15.680478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:15.680478 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:15.680478 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:15.680478 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:15.680478 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:15.680478 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:15.680478 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:15.680478 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:15.681487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:15.681487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:15.681487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:15.681487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:15.681487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:15.681487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:15.681487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:15.681487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:15.681487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:15.681487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:15.681487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:15.681487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:15.682073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:15.682073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:15.682073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:15.682073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:15.682073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:15.682073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:15.682073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:15.682073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:15.682073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:15.682073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:15.682073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:15.682073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:15.682073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:15.682632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:15.683327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:15.684131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:15.684131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:15.684131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:15.684131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:15.684131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:15.684131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:16.099758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:16.100402 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:16.100611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:16.101750 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:16.102086 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:16.102356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.103372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:16.103372 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.103372 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:16.103372 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:16.103372 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.103372 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:16.103372 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.103372 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:16.105136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.105136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:16.105136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:16.105136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:16.105136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:16.105136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:16.105136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:16.105136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.105136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:16.105136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:16.105136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:16.105136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.105533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:16.105533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:16.105533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:16.105533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:16.105533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:16.105533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:16.105533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:16.105533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:16.105533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.105533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:16.105533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:16.105533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.105533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:16.105894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.105894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:16.107172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:16.107727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.108266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:16.108266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.108266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:16.108266 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:16.108266 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:16.108266 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:16.108518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:32:16.108971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:16.109299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.110087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:16.110087 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.110087 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:16.110087 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:16.110087 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.110087 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:16.110087 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.110087 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:16.111585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:16.112420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:16.112420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:16.112420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.112420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:16.112420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:16.112420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.112420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:16.112420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.112420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:16.113120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.113120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:16.113120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:16.113120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:16.113120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:16.113120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:16.113120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:16.113120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:32:16.113120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:16.113120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:16.113120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.113120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:16.113515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:16.113913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.113913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:16.113913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.113913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:16.115074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:16.115589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:16.116070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.116070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:16.116070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:16.116070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.116070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:16.116070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.116070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:16.145124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:16.145647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:16.145875 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:16.146154 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:16.146667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:16.146910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.147557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:16.147557 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.147557 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:16.147557 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:16.147557 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.147557 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:16.147557 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.147557 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:16.149322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:16.149819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:16.149819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:16.149819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.149819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:16.149819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:16.149819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.149819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:16.149819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.149819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:16.150748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.150748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:16.150748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:16.150748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:16.150748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:16.150748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:16.150748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:16.150748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:16.150748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:16.150748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.150748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:16.151074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:16.151555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.151555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:16.151555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.151555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:16.152608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:16.153394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:16.154188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.154188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:16.154188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.154188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:16.155736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:16.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:16.156571 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:16.156732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:16.157143 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:16.157513 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:16.157711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.158417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:16.158417 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.158417 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:16.158417 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:16.158417 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.158417 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:16.158417 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.158417 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:16.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:16.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:16.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:16.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:16.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:16.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:16.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:16.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:16.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:16.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.159877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:16.159877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:16.159877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:16.159877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:16.159877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:16.159877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:16.159877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:16.159877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:16.159877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.159877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:16.159877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:16.159877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.159877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:16.160598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.160598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:16.160753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:16.161221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:16.161745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:16.161745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:16.161745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:20.643272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:20.644024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:20.644090 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:20.645527 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:20.680521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:20.680835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:20.681552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:20.681552 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:20.681552 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:20.681552 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:20.681552 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:20.681552 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:20.681552 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:20.681552 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:20.682272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:20.682272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:20.682272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:20.682272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:20.682272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:20.682272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:20.682272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:20.682272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:20.682272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:20.682272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:20.682272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:20.682272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:20.682864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:20.682864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:20.682864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:20.682864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:20.682864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:20.682864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:20.682864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:20.682864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:20.682864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:20.682864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:20.682864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:20.682864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:20.682864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:20.683498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:20.684171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:20.685039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:20.685039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:20.685039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:20.685039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:20.685039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:20.685039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:21.100471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:21.101822 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:21.101994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:21.102757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:21.103102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:21.103369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.107074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:21.107074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.107074 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:21.107074 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:21.107074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.107074 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:21.107074 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.107074 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:21.107956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.107956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:21.107956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:21.107956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:21.107956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:21.107956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:21.107956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:21.107956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.107956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:21.107956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:21.107956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:21.107956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.109187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:21.109187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:21.109187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:21.109187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:21.109187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:21.109187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:21.109187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:21.109187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:21.109187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.109187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:21.109187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:21.109187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.109187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:21.109801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:21.110466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.110915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:21.110915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:21.110915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.110915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:21.110915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.110915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:21.110915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:21.111106 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:21.111627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:21.111627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:32:21.111627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:21.111859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.113018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:21.113018 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.113018 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:21.113018 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:21.113018 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.113018 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:21.113018 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.113018 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:21.113745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:21.114730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:21.114730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:21.114730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.114730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:21.114730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:21.114730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.114730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:21.114730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.114730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:21.115373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.115373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:21.115373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:21.115373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:21.115373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:21.115373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:21.115373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:21.115373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:32:21.115373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:21.115373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:21.115373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.115373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:21.116183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:21.116940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:21.117304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:21.118239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:21.118239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:21.118239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:21.118239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:21.118239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:21.118239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.118239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:21.118239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:21.118239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.118239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:21.118239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.118239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:21.145742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:21.146756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:21.146861 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:21.147018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:21.147510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:21.147853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.149337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:21.149337 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.149337 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:21.149337 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:21.149337 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.149337 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:21.149337 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.149337 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:21.150183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:21.150841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:21.150841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:21.150841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.150841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:21.150841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:21.150841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.150841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:21.150841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.150841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:21.154033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.154033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:21.154033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:21.154033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:21.154033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:21.154033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:21.154033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:21.154033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:21.154033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:21.154033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.154033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:21.154523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:21.154845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.154845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:21.154845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.154845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:21.155773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:21.156264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:21.156733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.156733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:21.156733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.156733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:21.156733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:21.156883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:21.156923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:21.157349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:21.157683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:21.158093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:21.158329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.158930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:21.158930 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.158930 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:21.158930 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:21.158930 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.158930 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:21.158930 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.158930 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:21.159916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.159916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:21.159916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:21.159916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:21.159916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:21.159916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:21.159916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:21.159916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.159916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:21.159916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:21.159916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:21.159916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.161171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:21.161171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:21.161171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:21.161171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:21.161171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:21.161171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:21.161171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:21.161171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:21.161171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.161171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:21.161171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:21.161171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.161171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:21.161784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:21.162485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:21.164237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:21.164237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:21.164237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:21.164237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:21.164237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:21.164237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:25.643957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:25.644762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:25.644835 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:25.646396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:25.684615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:25.684816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:25.685349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:25.685349 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:25.685349 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:25.685349 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:25.685349 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:25.685349 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:25.685349 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:25.685349 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:25.686267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:25.686267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:25.686267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:25.686267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:25.686267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:25.686267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:25.686267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:25.686267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:25.686267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:25.686267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:25.686267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:25.686267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:25.686828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:25.686828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:25.686828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:25.686828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:25.686828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:25.686828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:25.686828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:25.686828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:25.686828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:25.686828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:25.686828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:25.686828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:25.686828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:25.687405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:25.688077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:25.688921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:25.688921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:25.688921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:25.688921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:25.688921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:25.688921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:26.101949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:26.102636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:26.102868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:26.104026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:26.104426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:26.104797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.105603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:26.105603 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.105603 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:26.105603 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:26.105603 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.105603 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:26.105603 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.105603 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:26.107147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.107147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:26.107147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:26.107147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:26.107147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:26.107147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:26.107147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:26.107147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.107147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:26.107147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:26.107147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:26.107147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.107651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:26.107651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:26.107651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:26.107651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:26.107651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:26.107651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:26.107651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:26.107651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:26.107651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.107651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:26.107651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:26.107651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.107651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:26.108004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.108004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:26.108907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:26.109387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.109916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:26.109916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.109916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:26.111389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:26.111815 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:26.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:26.112175 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:32:26.112604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:26.112831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.113326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:26.113326 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.113326 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:26.113326 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:26.113326 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.113326 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:26.113326 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.113326 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:26.114059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:26.114655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:26.114655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:26.114655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.114655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:26.114655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:26.114655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.114655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:26.114655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.114655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:26.114930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.114930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:26.114930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:26.114930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:26.114930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:26.114930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:26.114930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:26.114930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:32:26.114930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:26.114930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:26.114930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.114930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:26.115286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:26.115743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.115743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:26.115743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.115743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:26.115975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:26.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:26.117156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.117156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:26.117156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:26.117156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.117156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:26.117156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.117156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:26.146752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:26.147178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:26.147431 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:26.147626 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:26.148151 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:26.148657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.149467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:26.149467 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.149467 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:26.149467 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:26.149467 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.149467 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:26.149467 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.149467 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:26.150606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:26.151468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:26.151468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:26.151468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.151468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:26.151468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:26.151468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.151468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:26.151468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.151468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:26.152070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.152070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:26.152070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:26.152070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:26.152070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:26.152070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:26.152070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:26.152070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:26.152070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:26.152070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.152070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:26.152829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:26.153479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.153479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:26.153479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.153479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:26.153841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:26.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:26.155284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.155284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:26.155284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.155284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:26.156866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:26.157300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:26.157495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:26.157656 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:26.157880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:26.158074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:26.158231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.158670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:26.158670 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.158670 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:26.158670 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:26.158670 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.158670 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:26.158670 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.158670 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:26.159459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.159459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:26.159459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:26.159459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:26.159459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:26.159459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:26.159459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:26.159459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.159459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:26.159459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:26.159459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:26.159459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.160156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:26.160156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:26.160156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:26.160156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:26.160156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:26.160156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:26.160156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:26.160156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:26.160156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.160156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:26.160156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:26.160156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.160156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:26.161023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:26.161811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:26.162905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:26.162905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:26.162905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:26.162905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:26.162905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:26.162905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:30.644682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:30.645595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:30.645595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:30.646773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:30.677422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:30.677921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:30.678812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:30.678812 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:30.678812 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:30.678812 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:30.678812 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:30.678812 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:30.678812 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:30.678812 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:30.679539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:30.679539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:30.679539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:30.679539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:30.679539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:30.679539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:30.679539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:30.679539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:30.679539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:30.679539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:30.679539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:30.679539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:30.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:30.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:30.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:30.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:30.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:30.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:30.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:30.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:30.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:30.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:30.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:30.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:30.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:30.680796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:30.681614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:30.682490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:30.682490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:30.682490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:30.682490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:30.682490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:30.682490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:31.102539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:31.103367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:31.103594 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:31.104532 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:31.104729 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:31.104851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.105484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:31.105484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.105484 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:31.105484 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:31.105484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.105484 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:31.105484 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.105484 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:31.107272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.107272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:31.107272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:31.107272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:31.107272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:31.107272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:31.107272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:31.107272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.107272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:31.107272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:31.107272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:31.107272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.107694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:31.107694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:31.107694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:31.107694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:31.107694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:31.107694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:31.107694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:31.107694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:31.107694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.107694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:31.107694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:31.107694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.107694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:31.108318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.108318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:31.110062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.111364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:31.111364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.111364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:31.111891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:31.112425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:31.112663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:31.112935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:32:31.113406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:31.113619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.114324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:31.114324 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.114324 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:31.114324 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:31.114324 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.114324 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:31.114324 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.114324 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:31.115782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:31.116649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:31.116649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:31.116649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.116649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:31.116649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:31.116649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.116649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:31.116649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.116649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:31.117707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.117707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:31.117707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:31.117707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:31.117707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:31.117707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:31.117707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:31.117707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:32:31.117707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:31.117707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:31.117707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.117707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:31.118075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:31.118524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.118524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:31.118524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.118524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:31.119369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:31.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:31.120778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.120778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:31.120778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:31.120778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.120778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:31.120778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.120778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:31.147471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:31.148020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:31.148149 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:31.148357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:31.149166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:31.149823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.150976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:31.150976 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.150976 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:31.150976 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:31.150976 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.150976 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:31.150976 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.150976 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:31.152512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:31.152997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:31.152997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:31.152997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.152997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:31.152997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:31.152997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.152997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:31.152997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.152997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:31.154025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.154025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:31.154025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:31.154025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:31.154025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:31.154025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:31.154025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:31.154025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:31.154025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:31.154025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.154025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:31.154440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:31.154875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.154875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:31.154875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.154875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:31.155962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:31.156533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:31.157051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.157051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:31.157051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.157051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:31.157354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:31.157586 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:31.157936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:31.158228 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:31.158772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:31.158997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:31.159230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.160040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:31.160040 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.160040 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:31.160040 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:31.160040 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.160040 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:31.160040 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.160040 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:31.160923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.160923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:31.160923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:31.160923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:31.160923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:31.160923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:31.160923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:31.160923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.160923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:31.160923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:31.160923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:31.160923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.161365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:31.161365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:31.161365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:31.161365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:31.161365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:31.161365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:31.161365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:31.161365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:31.161365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.161365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:31.161365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:31.161365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.161365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:31.161803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.161803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:31.161990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:31.163042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:31.164375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:31.164375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:31.164375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:35.645362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:35.646260 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:35.646494 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:35.647979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:35.683718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:35.684154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:35.685103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:35.685103 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:35.685103 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:35.685103 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:35.685103 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:35.685103 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:35.685103 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:35.685103 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:35.686357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:35.686357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:35.686357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:35.686357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:35.686357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:35.686357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:35.686357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:35.686357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:35.686357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:35.686357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:35.686357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:35.686357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:35.686932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:35.686932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:35.686932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:35.686932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:35.686932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:35.686932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:35.686932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:35.686932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:35.686932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:35.686932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:35.686932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:35.686932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:35.686932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:35.687888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:35.689412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:35.690579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:35.690579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:35.690579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:35.690579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:35.690579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:35.690579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:36.103285 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:36.104139 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:36.104315 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:36.106968 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:36.106968 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:36.106968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.107265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:36.107265 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.107265 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:36.107265 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:36.107265 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.107265 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:36.107265 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.107265 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:36.108232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.108232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:36.108232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:36.108232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:36.108232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:36.108232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:36.108232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:36.108232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.108232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:36.108232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:36.108232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:36.108232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.108698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:36.108698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:36.108698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:36.108698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:36.108698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:36.108698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:36.108698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:36.108698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:36.108698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.108698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:36.108698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:36.108698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.108698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:36.109165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.109165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:36.109294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:36.109869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.110624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:36.110624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.110624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:36.112431 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:36.112844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:36.113001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:36.113135 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:32:36.114558 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:36.114558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.115170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:36.115170 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.115170 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:36.115170 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:36.115170 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.115170 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:36.115170 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.115170 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:36.116351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:36.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:36.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:36.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:36.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:36.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:36.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.117036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:36.118509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.118509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:36.118509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:36.118509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:36.118509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:36.118509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:36.118509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:36.118509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:32:36.118509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:36.118509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:36.118509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.118509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:36.119482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:36.120080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:36.120739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:36.121560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:36.121560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:36.121560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:36.121560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:36.121560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:36.121560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.121560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:36.121560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:36.121560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.121560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:36.121560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.121560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:36.148101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:36.148818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:36.148917 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:36.149107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:36.150052 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:36.150547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.151600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:36.151600 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.151600 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:36.151600 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:36.151600 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.151600 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:36.151600 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.151600 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:36.152960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:36.153684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:36.153684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:36.153684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.153684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:36.153684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:36.153684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.153684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:36.153684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.153684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:36.154067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.154067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:36.154067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:36.154067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:36.154067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:36.154067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:36.154067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:36.154067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:36.154067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:36.154067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.154067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:36.154632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:36.155358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.155358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:36.155358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.155358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:36.155610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:36.156358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:36.157121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.157121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:36.157121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.157121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:36.157710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:36.158058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:36.158200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:36.158494 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:36.159610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:36.159980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:36.160218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.161099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:36.161099 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.161099 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:36.161099 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:36.161099 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.161099 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:36.161099 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.161099 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:36.161994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.161994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:36.161994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:36.161994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:36.161994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:36.161994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:36.161994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:36.161994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.161994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:36.161994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:36.161994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:36.161994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:36.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:36.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:36.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:36.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:36.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:36.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:36.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:36.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:36.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:36.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:36.162983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.162983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:36.163278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:36.163959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:36.164488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:36.164488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:36.164488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:40.646079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:40.646724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:40.646959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:40.648371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:40.649979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:32:40.684534 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:40.684800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:40.685453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:40.685453 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:40.685453 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:40.685453 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:40.685453 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:40.685453 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:40.685453 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:40.685453 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:40.686315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:40.686315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:40.686315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:40.686315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:40.686315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:40.686315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:40.686315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:40.686315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:40.686315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:40.686315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:40.686315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:40.686315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:40.686738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:40.686738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:40.686738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:40.686738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:40.686738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:40.686738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:40.686738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:40.686738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:40.686738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:40.686738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:40.686738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:40.686738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:40.686738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:40.687182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:40.687182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:40.687365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:40.687735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:40.688153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:40.688153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:40.688153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:41.103858 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:41.104358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:41.104530 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:41.105438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:41.105728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:41.106070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.106986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:41.106986 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.106986 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:41.106986 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:41.106986 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.106986 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:41.106986 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.106986 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:41.108496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.108496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:41.108496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:41.108496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:41.108496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:41.108496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:41.108496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:41.108496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.108496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:41.108496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:41.108496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:41.108496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.108889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:41.108889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:41.108889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:41.108889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:41.108889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:41.108889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:41.108889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:41.108889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:41.108889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.108889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:41.108889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:41.108889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.108889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:41.109281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.109281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:41.110156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:41.110850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.111407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:41.111407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.111407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:41.112808 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:41.113150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:41.113439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:41.113694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:32:41.114051 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:41.114310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.114960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:41.114960 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.114960 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:41.114960 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:41.114960 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.114960 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:41.114960 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.114960 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:41.115775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:41.116401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:41.116401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:41.116401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.116401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:41.116401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:41.116401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.116401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:41.116401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.116401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:41.116934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.116934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:41.116934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:41.116934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:41.116934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:41.116934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:41.116934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:41.116934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:32:41.116934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:41.116934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:41.116934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.116934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:41.117327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:41.117719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.117719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:41.117719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.117719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:41.118472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:41.118999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:41.119571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.119571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:41.119571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:41.119571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.119571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:41.119571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.119571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:41.148678 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:41.149023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:41.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:41.149414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:41.149851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:41.150225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.150926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:41.150926 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.150926 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:41.150926 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:41.150926 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.150926 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:41.150926 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.150926 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:41.152087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:41.152813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:41.152813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:41.152813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.152813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:41.152813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:41.152813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.152813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:41.152813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.152813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:41.153443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.153443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:41.153443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:41.153443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:41.153443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:41.153443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:41.153443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:41.153443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:41.153443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:41.153443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.153443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:41.154000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:41.154679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.154679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:41.154679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.154679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:41.155052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:41.155947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:41.156893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.156893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:41.156893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.156893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:41.159479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:41.163952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:41.164045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:41.164156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:41.164667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:41.164919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:41.165072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.165583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:41.165583 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.165583 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:41.165583 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:41.165583 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.165583 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:41.165583 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.165583 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:41.166167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.166167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:41.166167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:41.166167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:41.166167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:41.166167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:41.166167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:41.166167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.166167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:41.166167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:41.166167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:41.166167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:41.166738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.166738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:41.166851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:41.167222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:41.167609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:41.167609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:41.167609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:45.646848 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:45.648126 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:45.648244 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:45.649012 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:45.688841 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:45.689227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:45.690075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:45.690075 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:45.690075 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:45.690075 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:45.690075 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:45.690075 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:45.690075 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:45.690075 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:45.690922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:45.690922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:45.690922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:45.690922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:45.690922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:45.690922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:45.690922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:45.690922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:45.690922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:45.690922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:45.690922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:45.690922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:45.691190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:45.691190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:45.691190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:45.691190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:45.691190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:45.691190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:45.691190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:45.691190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:45.691190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:45.691190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:45.691190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:45.691190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:45.691190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:45.691454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:45.691454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:45.691692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:45.692010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:45.692386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:45.692386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:45.692386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:46.105317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:46.105589 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:46.105782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:46.106965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:46.107392 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:46.107645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.108725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:46.108725 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.108725 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:46.108725 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:46.108725 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.108725 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:46.108725 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.108725 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:46.110450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.110450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:46.110450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:46.110450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:46.110450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:46.110450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:46.110450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:46.110450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.110450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:46.110450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:46.110450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:46.110450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.111494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:46.111494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:46.111494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:46.111494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:46.111494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:46.111494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:46.111494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:46.111494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:46.111494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.111494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:46.111494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:46.111494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.111494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:46.112176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.112176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:46.112320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:46.113065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.113918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:46.113918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.113918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:46.113918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:46.113918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:46.114156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:46.114609 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:32:46.114917 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:46.115302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.116336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:46.116336 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.116336 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:46.116336 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:46.116336 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.116336 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:46.116336 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.116336 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:46.117564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:46.118017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:46.118017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:46.118017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.118017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:46.118017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:46.118017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.118017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:46.118017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.118017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:46.118835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.118835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:46.118835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:46.118835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:46.118835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:46.118835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:46.118835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:46.118835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:32:46.118835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:46.118835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:46.118835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.118835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:46.119233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:46.119634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.119634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:46.119634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.119634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:46.120444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:46.120957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:46.121531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.121531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:46.121531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:46.121531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.121531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:46.121531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.121531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:46.149781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:46.150410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:46.150629 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:46.150888 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:46.151893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:46.152392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.153679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:46.153679 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.153679 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:46.153679 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:46.153679 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.153679 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:46.153679 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.153679 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:46.154943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:46.155654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:46.155654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:46.155654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.155654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:46.155654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:46.155654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.155654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:46.155654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.155654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:46.156459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.156459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:46.156459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:46.156459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:46.156459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:46.156459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:46.156459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:46.156459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:46.156459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:46.156459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.156459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:46.156979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:46.157647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.157647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:46.157647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.157647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:46.158170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:46.158984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:46.159987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.159987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:46.159987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.159987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:46.164583 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:46.164951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:46.165078 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:46.165163 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:46.166005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:46.166667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:46.166953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.168079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:46.168079 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.168079 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:46.168079 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:46.168079 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.168079 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:46.168079 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.168079 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:46.169122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.169122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:46.169122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:46.169122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:46.169122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:46.169122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:46.169122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:46.169122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.169122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:46.169122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:46.169122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:46.169122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.169841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:46.169841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:46.169841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:46.169841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:46.169841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:46.169841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:46.169841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:46.169841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:46.169841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.169841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:46.169841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:46.169841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.169841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:46.170441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:46.171156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:46.172038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:46.172038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:46.172038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:46.172038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:46.172038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:46.172038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:50.648540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:50.649389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:50.649389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:50.650646 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:50.681813 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:50.682380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:50.683406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:50.683406 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:50.683406 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:50.683406 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:50.683406 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:50.683406 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:50.683406 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:50.683406 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:50.684514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:50.684514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:50.684514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:50.684514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:50.684514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:50.684514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:50.684514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:50.684514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:50.684514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:50.684514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:50.684514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:50.684514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:50.685115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:50.685115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:50.685115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:50.685115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:50.685115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:50.685115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:50.685115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:50.685115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:50.685115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:50.685115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:50.685115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:50.685115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:50.685115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:50.685813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:50.686547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:50.687378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:50.687378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:50.687378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:50.687378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:50.687378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:50.687378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:51.105372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:51.105834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:51.105920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:51.106595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:51.106893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:51.107087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:51.110641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:51.111315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:51.112266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:51.113178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:51.114652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:51.114652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:51.114652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:51.114652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.114652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:51.114652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:51.114652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.114652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:51.114652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.114652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:51.114652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:51.115897 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:51.116467 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:51.117089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:32:51.117739 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:51.117988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.119235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.119773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:51.120771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.120771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:51.120771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:51.120771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:51.120771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:51.120771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:51.120771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:51.120771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:32:51.120771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:51.120771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:51.120771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.120771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:51.121123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:51.122342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:51.122711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:51.123112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:51.123112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:51.123112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:51.123112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:51.123112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:51.123112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.123112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:51.123112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:51.123112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.123112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:51.123112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.123112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:51.150373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:51.151485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:51.151485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:51.151485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:51.152119 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:51.152290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.153280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:51.153280 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.153280 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:51.153280 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:51.153280 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.153280 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:51.153280 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.153280 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:51.156135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:51.156681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:51.157264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:51.157264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:51.157264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:51.157264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:51.157264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.157264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:51.157264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:51.157264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:51.157264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.157264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:51.157264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:51.157264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:51.157264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:51.157805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:51.157805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:51.157805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:51.157805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:51.157805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.157805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:51.157805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:51.157805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.157805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:51.157805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.157805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:51.159119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:51.159823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:51.160595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.160595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:51.160595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.160595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:51.165038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:51.165525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:51.165525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:51.165525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:51.165827 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:51.166238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:51.166442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:51.170166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:51.171004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:51.171538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:51.171959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:51.172566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:51.172566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:51.172566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:51.172566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:51.172566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:51.172566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:51.172566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:51.172566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:51.172566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:51.172566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:55.649302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:55.649937 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:55.649937 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:55.650613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:55.680179 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:55.680374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:55.681924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:55.681924 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:55.681924 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:55.681924 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:55.681924 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:55.681924 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:55.681924 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:55.681924 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:55.682327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:55.682327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:55.682327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:55.682327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:55.682327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:55.682327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:55.682327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:55.682327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:55.682327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:55.682327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:55.682327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:55.682327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:55.683249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:55.683249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:55.683249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:55.683249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:55.683249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:55.683249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:55.683249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:55.683249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:55.683249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:55.683249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:55.683249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:55.683249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:55.683249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:55.683979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:55.684651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:55.685434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:55.685434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:55.685434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:55.685434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:55.685434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:55.685434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:56.106170 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:56.106673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:56.106751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:56.107562 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:56.107873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:56.108104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:56.109066 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.109066 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:56.109066 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:56.109066 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.109066 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:56.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.109066 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:56.109944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.109944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:56.109944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:56.109944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:56.109944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:56.109944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:56.109944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:56.109944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.109944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:56.109944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:56.109944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:56.109944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.110568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:56.110568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:56.110568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:56.110568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:56.110568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:56.110568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:56.110568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:56.110568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:56.110568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.110568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:56.110568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:56.110568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.110568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:56.111174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:56.111991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.112806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:56.112806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:56.112806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.112806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:56.112806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.112806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:56.116099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:56.116543 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:56.116655 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:56.116740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:32:56.117410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:56.117870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.118826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:56.118826 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.118826 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:56.118826 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:56.118826 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.118826 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:56.118826 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.118826 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:56.119562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:56.120184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:56.120184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:56.120184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.120184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:56.120184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:56.120184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.120184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:56.120184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.120184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:56.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:56.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:56.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:56.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:56.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:56.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:56.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:32:56.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:56.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:56.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.120578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:56.121030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:56.121533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:56.122161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:56.122858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:56.122858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:56.122858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:56.122858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:56.122858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:56.122858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.122858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:56.122858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:56.122858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.122858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:56.122858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.122858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:56.151049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:56.151742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:56.151990 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:56.152294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:56.152945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:56.153524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.154520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:56.154520 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.154520 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:56.154520 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:56.154520 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.154520 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:56.154520 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.154520 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:32:56.155944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:32:56.156965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:56.156965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:32:56.156965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.156965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:32:56.156965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:32:56.156965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.156965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:32:56.156965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.156965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:32:56.157816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.157816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:56.157816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:56.157816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:56.157816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:56.157816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:56.157816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:56.157816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:56.157816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:56.157816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.157816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:56.158514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:56.159015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.159015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:56.159015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.159015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:56.160045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:56.160706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:56.161402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.161402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:56.161402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.161402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:32:56.165436 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:32:56.166838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:32:56.166838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:32:56.166838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:32:56.166838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:56.167063 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:32:56.167150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.167824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:32:56.167824 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.167824 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:32:56.167824 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:32:56.167824 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.167824 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:32:56.167824 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.167824 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:32:56.169495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.169495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:32:56.169495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:56.169495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:32:56.169495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:32:56.169495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:56.169495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:56.169495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.169495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:56.169495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:56.169495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:32:56.169495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.169996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:32:56.169996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:56.169996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:32:56.169996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:56.169996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:32:56.169996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:32:56.169996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:56.169996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:32:56.169996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.169996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:32:56.169996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:32:56.169996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.169996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:32:56.170486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.170486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:32:56.170918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:32:56.171542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:32:56.172270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:32:56.172270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:32:56.172270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:00.650001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:00.650850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:00.650850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:00.651251 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:00.689329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:00.689574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:00.690221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:00.690221 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:00.690221 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:00.690221 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:00.690221 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:00.690221 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:00.690221 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:00.690221 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:00.691020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:00.691020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:00.691020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:00.691020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:00.691020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:00.691020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:00.691020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:00.691020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:00.691020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:00.691020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:00.691020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:00.691020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:00.691637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:00.691637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:00.691637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:00.691637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:00.691637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:00.691637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:00.691637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:00.691637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:00.691637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:00.691637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:00.691637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:00.691637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:00.691637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:00.692188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:00.692864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:00.693423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:00.693423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:00.693423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:00.693423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:00.693423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:00.693423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:01.106859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:01.107454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:01.107655 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:01.108692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:01.109522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:01.109522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.110338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:01.110338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.110338 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:01.110338 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:01.110338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.110338 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:01.110338 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.110338 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:01.111951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.111951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:01.111951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:01.111951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:01.111951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:01.111951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:01.111951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:01.111951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.111951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:01.111951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:01.111951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:01.111951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.112588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:01.112588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:01.112588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:01.112588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:01.112588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:01.112588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:01.112588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:01.112588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:01.112588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.112588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:01.112588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:01.112588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.112588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:01.113336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.113336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:01.113754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:01.114529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.115446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:01.115446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.115446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:01.116614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:01.116896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:01.116981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:01.117122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:33:01.117545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:01.117857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.118512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:01.118512 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.118512 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:01.118512 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:01.118512 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.118512 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:01.118512 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.118512 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:01.119190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:01.119886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:01.119886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:01.119886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.119886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:01.119886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:01.119886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.119886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:01.119886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.119886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:01.120403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.120403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:01.120403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:01.120403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:01.120403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:01.120403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:01.120403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:01.120403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:33:01.120403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:01.120403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:01.120403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.120403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:01.121162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:01.121689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:01.122065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:01.122589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:01.122589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:01.122589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:01.122589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:01.122589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:01.122589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.122589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:01.122589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:01.122589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.122589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:01.122589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.122589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:01.151975 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:01.152582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:01.152784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:01.152993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:01.153728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:01.154227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.155248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:01.155248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.155248 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:01.155248 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:01.155248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.155248 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:01.155248 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.155248 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:01.156583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:01.157122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:01.157122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:01.157122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.157122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:01.157122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:01.157122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.157122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:01.157122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.157122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:01.158027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.158027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:01.158027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:01.158027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:01.158027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:01.158027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:01.158027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:01.158027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:01.158027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:01.158027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.158027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:01.158486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:01.158941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.158941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:01.158941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.158941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:01.160008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:01.160685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:01.161395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.161395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:01.161395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.161395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:01.165868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:01.166107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:01.166225 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:01.166377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:01.166700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:01.166980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:01.167146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.167705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:01.167705 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.167705 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:01.167705 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:01.167705 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.167705 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:01.167705 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.167705 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:01.168623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.168623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:01.168623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:01.168623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:01.168623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:01.168623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:01.168623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:01.168623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.168623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:01.168623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:01.168623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:01.168623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.169080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:01.169080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:01.169080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:01.169080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:01.169080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:01.169080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:01.169080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:01.169080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:01.169080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.169080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:01.169080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:01.169080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.169080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:01.169570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.169570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:01.169656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:01.170229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:01.171031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:01.171031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:01.171031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:05.650788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:05.651623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:05.651623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:05.653046 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:05.695129 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:05.695644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:05.696604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:05.696604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:05.696604 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:05.696604 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:05.696604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:05.696604 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:05.696604 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:05.696604 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:05.698035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:05.698035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:05.698035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:05.698035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:05.698035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:05.698035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:05.698035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:05.698035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:05.698035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:05.698035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:05.698035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:05.698035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:05.698712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:05.698712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:05.698712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:05.698712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:05.698712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:05.698712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:05.698712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:05.698712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:05.698712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:05.698712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:05.698712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:05.698712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:05.698712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:05.699436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:05.699436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:05.699549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:05.700572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:05.701582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:05.701582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:05.701582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:06.107580 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:06.108196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:06.108424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:06.109400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:06.109757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:06.109996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.110929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:06.110929 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.110929 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:06.110929 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:06.110929 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.110929 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:06.110929 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.110929 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:06.112446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.112446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:06.112446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:06.112446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:06.112446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:06.112446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:06.112446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:06.112446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.112446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:06.112446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:06.112446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:06.112446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.113039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:06.113039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:06.113039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:06.113039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:06.113039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:06.113039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:06.113039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:06.113039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:06.113039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.113039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:06.113039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:06.113039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.113039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:06.113708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.113708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:06.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:06.114844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.115715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:06.115715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.115715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:06.117050 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:06.117364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:06.117502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:06.117625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:33:06.118043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:06.118362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.119033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:06.119033 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.119033 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:06.119033 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:06.119033 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.119033 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:06.119033 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.119033 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:06.119836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:06.120318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:06.120318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:06.120318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.120318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:06.120318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:06.120318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.120318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:06.120318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.120318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:06.120690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.120690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:06.120690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:06.120690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:06.120690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:06.120690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:06.120690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:06.120690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:33:06.120690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:06.120690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:06.120690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.120690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:06.121037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:06.121468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.121468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:06.121468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.121468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:06.121780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:06.122251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:06.122740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.122740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:06.122740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:06.122740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.122740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:06.122740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.122740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:06.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:06.152840 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:06.153014 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:06.153243 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:06.153638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:06.153874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.154478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:06.154478 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.154478 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:06.154478 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:06.154478 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.154478 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:06.154478 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.154478 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:06.155628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:06.156022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:06.156022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:06.156022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.156022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:06.156022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:06.156022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.156022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:06.156022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.156022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:06.156963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.156963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:06.156963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:06.156963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:06.156963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:06.156963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:06.156963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:06.156963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:06.156963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:06.156963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.156963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:06.157293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:06.157704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.157704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:06.157704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.157704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:06.158879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:06.159383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:06.159851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.159851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:06.159851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.159851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:06.166309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:06.166613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:06.166730 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:06.166885 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:06.167307 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:06.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:06.167896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.168606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:06.168606 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.168606 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:06.168606 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:06.168606 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.168606 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:06.168606 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.168606 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:06.169540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.169540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:06.169540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:06.169540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:06.169540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:06.169540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:06.169540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:06.169540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.169540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:06.169540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:06.169540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:06.169540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.169909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:06.169909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:06.169909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:06.169909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:06.169909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:06.169909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:06.169909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:06.169909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:06.169909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.169909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:06.169909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:06.169909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.169909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:06.170261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.170261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:06.170970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:06.171666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:06.172513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:06.172513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:06.172513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:10.651887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:10.652834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:10.652834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:10.653833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:10.685898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:10.686389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:10.687418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:10.687418 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:10.687418 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:10.687418 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:10.687418 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:10.687418 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:10.687418 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:10.687418 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:10.688506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:10.688506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:10.688506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:10.688506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:10.688506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:10.688506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:10.688506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:10.688506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:10.688506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:10.688506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:10.688506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:10.688506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:10.688893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:10.688893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:10.688893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:10.688893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:10.688893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:10.688893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:10.688893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:10.688893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:10.688893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:10.688893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:10.688893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:10.688893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:10.688893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:10.689384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:10.689384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:10.689570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:10.689974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:10.690512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:10.690512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:10.690512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:11.108316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:11.108701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:11.108797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:11.109490 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:11.110154 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:11.110154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.110888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:11.110888 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.110888 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:11.110888 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:11.110888 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.110888 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:11.110888 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.110888 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:11.111735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.111735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:11.111735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:11.111735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:11.111735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:11.111735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:11.111735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:11.111735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.111735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:11.111735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:11.111735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:11.111735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.112120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:11.112120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:11.112120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:11.112120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:11.112120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:11.112120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:11.112120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:11.112120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:11.112120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.112120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:11.112120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:11.112120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.112120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:11.112484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.112484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:11.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:11.114989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.115506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:11.115506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.115506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:11.117673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:11.118092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:11.118092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:11.118274 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:33:11.118893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:11.119157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.120729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:11.120729 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.120729 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:11.120729 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:11.120729 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.120729 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:11.120729 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.120729 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:11.121561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:11.121997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:11.121997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:11.121997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.121997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:11.121997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:11.121997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.121997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:11.121997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.121997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:11.122436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.122436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:11.122436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:11.122436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:11.122436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:11.122436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:11.122436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:11.122436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:33:11.122436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:11.122436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:11.122436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.122436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:11.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:11.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:11.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.123163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:11.123907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:11.124374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:11.124862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.124862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:11.124862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:11.124862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.124862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:11.124862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.124862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:11.152874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:11.153507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:11.153507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:11.153596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:11.154061 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:11.154519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.156456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.157008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:11.157397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.157397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:11.157397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:11.157397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:11.157397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:11.157397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:11.157397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:11.157397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:11.157397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:11.157397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.157397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:11.157686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:11.158050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.158050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:11.158050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.158050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:11.158918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:11.159390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:11.160129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.160129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:11.160129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.160129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:11.167528 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:11.168120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:11.168120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:11.168120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:11.170942 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:11.171354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:11.171438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.172261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:11.172261 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.172261 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:11.172261 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:11.172261 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.172261 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:11.172261 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.172261 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:11.173581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.173581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:11.173581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:11.173581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:11.173581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:11.173581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:11.173581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:11.173581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.173581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:11.173581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:11.173581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:11.173581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.173966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:11.173966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:11.173966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:11.173966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:11.173966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:11.173966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:11.173966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:11.173966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:11.173966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.173966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:11.173966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:11.173966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.173966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:11.174448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.174448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:11.175029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:11.175487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:11.175966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:11.175966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:11.175966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:15.652384 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:15.655468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:15.655468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:15.655468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:15.693655 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:15.693980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:15.694806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:15.694806 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:15.694806 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:15.694806 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:15.694806 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:15.694806 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:15.694806 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:15.694806 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:15.697218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:15.697218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:15.697218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:15.697218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:15.697218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:15.697218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:15.697218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:15.697218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:15.697218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:15.697218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:15.697218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:15.697218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:15.698448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:15.698448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:15.698448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:15.698448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:15.698448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:15.698448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:15.698448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:15.698448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:15.698448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:15.698448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:15.698448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:15.698448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:15.698448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:15.699428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:15.700454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:15.700943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:15.700943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:15.700943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:15.700943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:15.700943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:15.700943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:16.108976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:16.109666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:16.109929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:16.110982 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:16.111250 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:16.111374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.112100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:16.112100 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.112100 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:16.112100 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:16.112100 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.112100 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:16.112100 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.112100 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:16.113747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.113747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:16.113747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:16.113747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:16.113747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:16.113747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:16.113747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:16.113747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.113747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:16.113747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:16.113747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:16.113747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.114427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:16.114427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:16.114427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:16.114427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:16.114427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:16.114427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:16.114427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:16.114427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:16.114427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.114427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:16.114427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:16.114427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.114427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:16.115022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.115022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:16.115818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:16.116755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.117627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:16.117627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.117627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:16.118100 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:16.118529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:16.118726 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:16.118924 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:33:16.119324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:16.119535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.120125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:16.120125 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.120125 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:16.120125 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:16.120125 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.120125 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:16.120125 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.120125 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:16.121097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:16.121633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:16.121633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:16.121633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.121633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:16.121633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:16.121633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.121633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:16.121633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.121633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:16.122278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.122278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:16.122278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:16.122278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:16.122278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:16.122278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:16.122278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:16.122278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:33:16.122278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:16.122278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:16.122278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.122278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:16.122680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:16.123117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.123117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:16.123117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.123117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:16.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:16.124315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:16.124917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.124917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:16.124917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:16.124917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.124917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:16.124917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.124917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:16.153702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:16.154356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:16.154560 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:16.154780 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:16.155292 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:16.155811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.156623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:16.156623 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.156623 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:16.156623 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:16.156623 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.156623 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:16.156623 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.156623 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:16.158060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:16.158763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:16.158763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:16.158763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.158763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:16.158763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:16.158763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.158763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:16.158763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.158763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:16.159711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.159711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:16.159711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:16.159711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:16.159711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:16.159711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:16.159711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:16.159711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:16.159711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:16.159711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.159711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:16.160094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:16.160539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.160539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:16.160539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.160539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:16.161701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:16.162283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:16.162848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.162848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:16.162848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.162848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:16.167977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:16.168383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:16.168519 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:16.168687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:16.169039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:16.169308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:16.169458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.170014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:16.170014 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.170014 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:16.170014 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:16.170014 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.170014 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:16.170014 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.170014 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:16.170883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.170883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:16.170883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:16.170883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:16.170883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:16.170883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:16.170883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:16.170883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.170883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:16.170883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:16.170883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:16.170883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.171500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:16.171500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:16.171500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:16.171500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:16.171500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:16.171500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:16.171500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:16.171500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:16.171500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.171500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:16.171500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:16.171500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.171500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:16.172134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:16.172832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:16.173743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:16.173743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:16.173743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:16.173743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:16.173743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:16.173743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:20.654166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:20.654771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:20.655461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:20.656131 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:20.691062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:20.691429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:20.692412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:20.692412 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:20.692412 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:20.692412 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:20.692412 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:20.692412 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:20.692412 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:20.692412 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:20.693228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:20.693228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:20.693228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:20.693228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:20.693228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:20.693228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:20.693228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:20.693228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:20.693228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:20.693228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:20.693228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:20.693228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:20.693845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:20.693845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:20.693845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:20.693845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:20.693845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:20.693845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:20.693845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:20.693845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:20.693845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:20.693845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:20.693845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:20.693845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:20.693845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:20.694528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:20.695350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:20.696317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:20.696317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:20.696317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:20.696317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:20.696317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:20.696317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:21.109949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:21.110508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:21.110710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:21.111754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:21.112041 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:21.112198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.113183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:21.113183 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.113183 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:21.113183 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:21.113183 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.113183 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:21.113183 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.113183 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:21.114712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.114712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:21.114712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:21.114712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:21.114712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:21.114712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:21.114712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:21.114712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.114712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:21.114712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:21.114712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:21.114712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.115305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:21.115305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:21.115305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:21.115305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:21.115305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:21.115305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:21.115305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:21.115305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:21.115305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.115305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:21.115305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:21.115305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.115305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:21.115967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.115967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:21.116658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:21.117434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.118306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:21.118306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.118306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:21.118478 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:21.118606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:21.118678 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:21.118821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:33:21.119319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:21.119654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.120315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:21.120315 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.120315 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:21.120315 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:21.120315 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.120315 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:21.120315 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.120315 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:21.121068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:21.121814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:21.121814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:21.121814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.121814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:21.121814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:21.121814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.121814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:21.121814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.121814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:21.122378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.122378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:21.122378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:21.122378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:21.122378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:21.122378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:21.122378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:21.122378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:33:21.122378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:21.122378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:21.122378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.122378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:21.122931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:21.123614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:21.124332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:21.125321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:21.125321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:21.125321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:21.125321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:21.125321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:21.125321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.125321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:21.125321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:21.125321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.125321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:21.125321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.125321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:21.154266 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:21.154614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:21.154786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:21.154970 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:21.158711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:21.158957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.159579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:21.159579 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.159579 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:21.159579 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:21.159579 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.159579 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:21.159579 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.159579 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:21.160845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:21.161558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:21.161558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:21.161558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.161558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:21.161558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:21.161558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.161558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:21.161558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.161558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:21.162191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.162191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:21.162191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:21.162191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:21.162191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:21.162191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:21.162191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:21.162191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:21.162191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:21.162191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.162191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:21.162865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:21.163528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.163528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:21.163528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.163528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:21.164227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:21.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:21.165958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.165958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:21.165958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.165958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:21.168436 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:21.168772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:21.168983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:21.169181 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:21.169657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:21.170090 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:21.170419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.171327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:21.171327 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.171327 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:21.171327 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:21.171327 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.171327 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:21.171327 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.171327 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:21.172182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.172182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:21.172182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:21.172182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:21.172182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:21.172182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:21.172182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:21.172182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.172182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:21.172182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:21.172182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:21.172182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.172718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:21.172718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:21.172718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:21.172718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:21.172718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:21.172718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:21.172718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:21.172718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:21.172718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.172718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:21.172718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:21.172718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.172718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:21.173346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:21.173888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:21.174637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:21.174637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:21.174637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:21.174637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:21.174637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:21.174637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:25.654983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:25.655816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:25.655816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:25.657179 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:25.700721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:25.701103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:25.702224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:25.702224 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:25.702224 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:25.702224 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:25.702224 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:25.702224 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:25.702224 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:25.702224 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:25.702927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:25.702927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:25.702927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:25.702927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:25.702927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:25.702927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:25.702927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:25.702927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:25.702927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:25.702927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:25.702927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:25.702927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:25.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:25.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:25.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:25.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:25.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:25.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:25.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:25.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:25.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:25.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:25.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:25.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:25.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:25.704273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:25.705043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:25.706019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:25.706019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:25.706019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:25.706019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:25.706019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:25.706019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:26.110630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:26.111110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:26.111455 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:26.112528 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:26.112869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:26.113112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.114070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:26.114070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.114070 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:26.114070 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:26.114070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.114070 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:26.114070 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.114070 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:26.115852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.115852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:26.115852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:26.115852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:26.115852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:26.115852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:26.115852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:26.115852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.115852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:26.115852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:26.115852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:26.115852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.116518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:26.116518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:26.116518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:26.116518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:26.116518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:26.116518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:26.116518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:26.116518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:26.116518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.116518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:26.116518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:26.116518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.116518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:26.117057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.117057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:26.117455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:26.118166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.118974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:26.118974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.118974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:26.118974 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:26.119171 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:26.119262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:26.119479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:33:26.119978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:26.120349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.121171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:26.121171 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.121171 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:26.121171 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:26.121171 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.121171 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:26.121171 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.121171 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:26.122483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:26.123504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:26.123504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:26.123504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.123504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:26.123504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:26.123504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.123504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:26.123504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.123504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:26.124157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.124157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:26.124157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:26.124157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:26.124157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:26.124157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:26.124157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:26.124157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:33:26.124157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:26.124157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:26.124157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.124157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:26.124589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:26.124954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.124954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:26.124954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.124954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:26.126027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:26.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:26.126974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.126974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:26.126974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:26.126974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.126974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:26.126974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.126974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:26.154839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:26.155321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:26.155507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:26.155718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:26.157272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.158387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:26.158387 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.158387 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:26.158387 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:26.158387 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.158387 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:26.158387 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.158387 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:26.159811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:26.160718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:26.160718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:26.160718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.160718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:26.160718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:26.160718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.160718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:26.160718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.160718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:26.161776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.161776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:26.161776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:26.161776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:26.161776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:26.161776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:26.161776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:26.161776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:26.161776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:26.161776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.161776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:26.162454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:26.163093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.163093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:26.163093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.163093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:26.164220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:26.165068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:26.166014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.166014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:26.166014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.166014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:26.168721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:26.168959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:26.169089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:26.169304 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:26.169678 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:26.170036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:26.170180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.170837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:26.170837 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.170837 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:26.170837 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:26.170837 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.170837 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:26.170837 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.170837 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:26.171835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.171835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:26.171835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:26.171835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:26.171835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:26.171835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:26.171835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:26.171835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.171835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:26.171835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:26.171835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:26.171835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.172475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:26.172475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:26.172475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:26.172475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:26.172475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:26.172475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:26.172475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:26.172475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:26.172475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.172475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:26.172475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:26.172475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.172475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:26.173071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:26.173808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:26.174508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:26.174508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:26.174508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:26.174508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:26.174508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:26.174508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:30.655675 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:30.656298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:30.656417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:30.657600 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:30.686064 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:30.686517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:30.687419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:30.687419 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:30.687419 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:30.687419 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:30.687419 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:30.687419 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:30.687419 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:30.687419 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:30.688186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:30.688186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:30.688186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:30.688186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:30.688186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:30.688186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:30.688186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:30.688186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:30.688186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:30.688186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:30.688186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:30.688186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:30.688568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:30.688568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:30.688568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:30.688568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:30.688568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:30.688568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:30.688568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:30.688568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:30.688568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:30.688568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:30.688568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:30.688568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:30.688568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:30.688937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:30.689391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:30.689920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:30.689920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:30.689920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:30.689920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:30.689920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:30.689920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:31.111320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:31.111933 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:31.112122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:31.113665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:31.113908 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:31.114067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.114757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:31.114757 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.114757 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:31.114757 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:31.114757 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.114757 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:31.114757 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.114757 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:31.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:31.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:31.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:31.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:31.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:31.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:31.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:31.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:31.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:31.116558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:31.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:31.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:31.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:31.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:31.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:31.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:31.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:31.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:31.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:31.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.117101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:31.117814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.117814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:31.118236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:31.119309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.120420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:31.120420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.120420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:31.120420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:31.120420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:31.120910 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:31.120910 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:33:31.121476 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:31.122012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.124955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:31.124955 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.124955 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:31.124955 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:31.124955 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.124955 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:31.124955 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.124955 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:31.126233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:31.126682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:31.126682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:31.126682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.126682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:31.126682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:31.126682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.126682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:31.126682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.126682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:31.127701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.127701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:31.127701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:31.127701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:31.127701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:31.127701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:31.127701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:31.127701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:33:31.127701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:31.127701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:31.127701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.127701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:31.128107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:31.128576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.128576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:31.128576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.128576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:31.129472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:31.129967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:31.130525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.130525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:31.130525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:31.130525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.130525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:31.130525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.130525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:31.155471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:31.155886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:31.156058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:31.156282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:31.156843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:31.157111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.157798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:31.157798 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.157798 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:31.157798 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:31.157798 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.157798 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:31.157798 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.157798 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:31.158989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:31.159834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:31.159834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:31.159834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.159834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:31.159834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:31.159834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.159834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:31.159834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.159834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:31.160193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.160193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:31.160193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:31.160193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:31.160193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:31.160193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:31.160193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:31.160193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:31.160193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:31.160193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.160193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:31.160663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:31.161464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:31.162044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:31.162867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:31.162867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:31.162867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:31.162867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:31.162867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.162867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:31.162867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:31.162867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.162867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:31.162867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.162867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:31.169260 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:31.169717 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:31.169875 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:31.170077 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:31.170594 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:31.170900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:31.171104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.171722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:31.171722 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.171722 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:31.171722 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:31.171722 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.171722 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:31.171722 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.171722 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:31.172800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.172800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:31.172800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:31.172800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:31.172800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:31.172800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:31.172800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:31.172800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.172800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:31.172800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:31.172800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:31.172800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.173146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:31.173146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:31.173146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:31.173146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:31.173146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:31.173146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:31.173146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:31.173146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:31.173146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.173146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:31.173146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:31.173146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.173146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:31.173514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.173514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:31.174086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:31.174535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:31.175039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:31.175039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:31.175039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:35.656371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:35.657072 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:35.657072 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:35.657922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:35.689889 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:35.690325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:35.691174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:35.691174 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:35.691174 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:35.691174 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:35.691174 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:35.691174 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:35.691174 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:35.691174 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:35.692063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:35.692063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:35.692063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:35.692063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:35.692063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:35.692063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:35.692063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:35.692063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:35.692063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:35.692063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:35.692063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:35.692063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:35.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:35.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:35.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:35.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:35.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:35.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:35.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:35.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:35.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:35.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:35.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:35.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:35.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:35.693273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:35.693906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:35.694728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:35.694728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:35.694728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:35.694728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:35.694728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:35.694728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:36.112127 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:36.112686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:36.112949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:36.114137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:36.114387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:36.114551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.115343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:36.115343 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.115343 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:36.115343 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:36.115343 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.115343 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:36.115343 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.115343 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:36.117009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.117009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:36.117009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:36.117009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:36.117009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:36.117009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:36.117009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:36.117009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.117009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:36.117009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:36.117009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:36.117009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.117655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:36.117655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:36.117655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:36.117655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:36.117655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:36.117655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:36.117655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:36.117655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:36.117655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.117655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:36.117655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:36.117655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.117655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:36.118282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.118282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:36.119124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:36.119910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.120734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:36.120734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.120734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:36.120734 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:36.120734 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:36.120734 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:36.120734 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:33:36.121356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:36.121739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.122710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:36.122710 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.122710 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:36.122710 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:36.122710 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.122710 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:36.122710 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.122710 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:36.123508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:36.124256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:36.124256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:36.124256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.124256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:36.124256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:36.124256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.124256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:36.124256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.124256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:36.124530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.124530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:36.124530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:36.124530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:36.124530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:36.124530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:36.124530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:36.124530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:33:36.124530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:36.124530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:36.124530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.124530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:36.124884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:36.125291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.125291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:36.125291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.125291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:36.125449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:36.125891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:36.126432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.126432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:36.126432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:36.126432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.126432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:36.126432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.126432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:36.156236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:36.156737 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:36.156925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:36.157120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:36.157956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:36.158428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.159273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:36.159273 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.159273 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:36.159273 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:36.159273 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.159273 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:36.159273 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.159273 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:36.160732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:36.161470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:36.161470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:36.161470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.161470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:36.161470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:36.161470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.161470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:36.161470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.161470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:36.162522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.162522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:36.162522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:36.162522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:36.162522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:36.162522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:36.162522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:36.162522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:36.162522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:36.162522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.162522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:36.163130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:36.163783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.163783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:36.163783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.163783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:36.165080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:36.165975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:36.166937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.166937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:36.166937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.166937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:36.169947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:36.170312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:36.170409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:36.170597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:36.171144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:36.171574 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:36.171803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.172561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:36.172561 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.172561 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:36.172561 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:36.172561 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.172561 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:36.172561 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.172561 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:36.173429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.173429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:36.173429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:36.173429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:36.173429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:36.173429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:36.173429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:36.173429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.173429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:36.173429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:36.173429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:36.173429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.173806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:36.173806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:36.173806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:36.173806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:36.173806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:36.173806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:36.173806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:36.173806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:36.173806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.173806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:36.173806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:36.173806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.173806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:36.174175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.174175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:36.174436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:36.174928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:36.175476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:36.175476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:36.175476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:40.657169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:40.657976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:40.657976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:40.659128 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:40.661590 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:33:40.689657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:40.690019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:40.690867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:40.690867 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:40.690867 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:40.690867 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:40.690867 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:40.690867 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:40.690867 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:40.690867 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:40.691639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:40.691639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:40.691639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:40.691639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:40.691639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:40.691639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:40.691639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:40.691639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:40.691639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:40.691639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:40.691639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:40.691639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:40.692261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:40.692261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:40.692261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:40.692261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:40.692261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:40.692261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:40.692261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:40.692261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:40.692261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:40.692261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:40.692261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:40.692261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:40.692261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:40.692817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:40.693536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:40.694346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:40.694346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:40.694346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:40.694346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:40.694346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:40.694346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:41.112774 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:41.113413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:41.113525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:41.114495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:41.114830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:41.115052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.115837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:41.115837 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.115837 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:41.115837 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:41.115837 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.115837 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:41.115837 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.115837 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:41.116930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.116930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:41.116930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:41.116930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:41.116930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:41.116930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:41.116930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:41.116930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.116930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:41.116930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:41.116930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:41.116930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.117618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:41.117618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:41.117618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:41.117618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:41.117618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:41.117618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:41.117618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:41.117618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:41.117618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.117618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:41.117618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:41.117618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.117618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:41.118199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.118199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:41.118982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:41.119709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.120597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:41.120597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.120597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:41.120597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:41.120789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:41.120912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:41.120977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:33:41.121383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:41.121580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.122684 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:41.123115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:41.123972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:41.123972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:41.123972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.123972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:41.123972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:41.123972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.123972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:41.123972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.123972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:41.124466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.124466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:41.124466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:41.124466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:41.124466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:41.124466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:41.124466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:41.124466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:33:41.124466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:41.124466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:41.124466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.124466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:41.125093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:41.125718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.125718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:41.125718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.125718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:41.127810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:41.128160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:41.128546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.128546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:41.128546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:41.128546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.128546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:41.128546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.128546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:41.156862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:41.157748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:41.157871 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:41.158018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:41.158700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:41.159121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.160037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:41.160037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.160037 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:41.160037 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:41.160037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.160037 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:41.160037 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.160037 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:41.161307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:41.161989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:41.161989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:41.161989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.161989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:41.161989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:41.161989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.161989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:41.161989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.161989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:41.167204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.167204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:41.167204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:41.167204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:41.167204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:41.167204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:41.167204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:41.167204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:41.167204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:41.167204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.167204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:41.167590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:41.168011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.168011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:41.168011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.168011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:41.170149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:41.170688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:41.171282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.171282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:41.171282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.171282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:41.171282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:41.171581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:41.171679 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:41.171842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:41.172436 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:41.172870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:41.173100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.173881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:41.173881 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.173881 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:41.173881 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:41.173881 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.173881 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:41.173881 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.173881 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:41.174710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.174710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:41.174710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:41.174710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:41.174710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:41.174710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:41.174710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:41.174710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.174710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:41.174710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:41.174710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:41.174710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.175199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:41.175199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:41.175199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:41.175199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:41.175199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:41.175199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:41.175199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:41.175199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:41.175199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.175199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:41.175199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:41.175199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.175199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:41.175725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:41.176395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:41.177423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:41.177423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:41.177423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:41.177423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:41.177423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:41.177423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:45.657862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:45.659450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:45.659450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:45.659450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:45.691820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:45.692134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:45.692968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:45.692968 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:45.692968 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:45.692968 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:45.692968 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:45.692968 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:45.692968 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:45.692968 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:45.693825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:45.693825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:45.693825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:45.693825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:45.693825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:45.693825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:45.693825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:45.693825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:45.693825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:45.693825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:45.693825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:45.693825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:45.694489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:45.694489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:45.694489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:45.694489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:45.694489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:45.694489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:45.694489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:45.694489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:45.694489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:45.694489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:45.694489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:45.694489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:45.694489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:45.695142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:45.695927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:45.696871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:45.696871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:45.696871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:45.696871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:45.696871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:45.696871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:46.113525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:46.114066 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:46.114315 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:46.115950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:46.116311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:46.116551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.117405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:46.117405 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.117405 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:46.117405 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:46.117405 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.117405 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:46.117405 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.117405 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:46.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:46.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:46.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:46.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:46.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:46.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:46.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:46.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:46.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:46.118876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.119496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:46.119496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:46.119496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:46.119496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:46.119496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:46.119496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:46.119496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:46.119496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:46.119496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.119496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:46.119496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:46.119496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.119496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:46.120085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.120085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:46.120524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:46.121380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.122434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:46.122434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.122434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:46.122434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:46.122434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:46.122434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:46.122434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:33:46.122434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:46.122837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.123911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:46.123911 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.123911 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:46.123911 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:46.123911 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.123911 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:46.123911 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.123911 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:46.125085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:46.125544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:46.125544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:46.125544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.125544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:46.125544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:46.125544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.125544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:46.125544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.125544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:46.126461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.126461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:46.126461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:46.126461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:46.126461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:46.126461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:46.126461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:46.126461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:33:46.126461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:46.126461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:46.126461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.126461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:46.126816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:46.127333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.127333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:46.127333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.127333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:46.127696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:46.128120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:46.128630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.128630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:46.128630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:46.128630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.128630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:46.128630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.128630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:46.157840 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:46.158328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:46.158506 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:46.158691 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:46.159262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:46.159664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.160452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:46.160452 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.160452 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:46.160452 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:46.160452 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.160452 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:46.160452 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.160452 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:46.161593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:46.162392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:46.162392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:46.162392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.162392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:46.162392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:46.162392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.162392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:46.162392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.162392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:46.162962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.162962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:46.162962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:46.162962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:46.162962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:46.162962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:46.162962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:46.162962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:46.162962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:46.162962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.162962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:46.163543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:46.164396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.164396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:46.164396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.164396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:46.164677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:46.165698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:46.166763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.166763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:46.166763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.166763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:46.171724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:46.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:46.172150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:46.172336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:46.172771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:46.173137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:46.173454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.174146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:46.174146 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.174146 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:46.174146 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:46.174146 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.174146 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:46.174146 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.174146 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:46.175042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.175042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:46.175042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:46.175042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:46.175042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:46.175042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:46.175042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:46.175042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.175042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:46.175042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:46.175042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:46.175042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.175687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:46.175687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:46.175687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:46.175687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:46.175687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:46.175687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:46.175687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:46.175687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:46.175687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.175687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:46.175687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:46.175687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.175687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:46.176294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:46.176931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:46.177767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:46.177767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:46.177767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:46.177767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:46.177767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:46.177767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:50.658663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:50.659691 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:50.659691 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:50.660924 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:50.697870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:50.698278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:50.699091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:50.699091 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:50.699091 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:50.699091 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:50.699091 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:50.699091 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:50.699091 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:50.699091 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:50.700192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:50.700192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:50.700192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:50.700192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:50.700192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:50.700192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:50.700192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:50.700192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:50.700192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:50.700192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:50.700192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:50.700192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:50.700787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:50.700787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:50.700787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:50.700787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:50.700787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:50.700787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:50.700787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:50.700787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:50.700787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:50.700787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:50.700787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:50.700787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:50.700787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:50.701421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:50.701421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:50.701749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:50.702330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:50.702905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:50.702905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:50.702905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:51.114192 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:51.114694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:51.114778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:51.115649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:51.116312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:51.116565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.117262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:51.117262 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.117262 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:51.117262 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:51.117262 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.117262 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:51.117262 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.117262 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:51.118138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.118138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:51.118138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:51.118138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:51.118138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:51.118138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:51.118138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:51.118138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.118138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:51.118138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:51.118138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:51.118138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.119366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:51.119366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:51.119366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:51.119366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:51.119366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:51.119366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:51.119366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:51.119366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:51.119366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.119366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:51.119366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:51.119366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.119366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:51.120048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:51.120783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:51.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:51.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:51.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:51.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:51.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:51.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:51.121772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:33:51.123124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:51.123714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.124799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:51.124799 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.124799 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:51.124799 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:51.124799 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.124799 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:51.124799 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.124799 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:51.125530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:51.126024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:51.126024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:51.126024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.126024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:51.126024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:51.126024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.126024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:51.126024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.126024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:51.126332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.126332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:51.126332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:51.126332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:51.126332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:51.126332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:51.126332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:51.126332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:33:51.126332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:51.126332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:51.126332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.126332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:51.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:51.127071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.127071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:51.127071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.127071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:51.127262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:51.127777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:51.128278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.128278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:51.128278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:51.128278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.128278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:51.128278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.128278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:51.158507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:51.158824 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:51.158929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:51.159067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:51.159581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:51.159828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.160376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:51.160376 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.160376 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:51.160376 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:51.160376 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.160376 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:51.160376 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.160376 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:51.161227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:51.161932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:51.161932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:51.161932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.161932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:51.161932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:51.161932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.161932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:51.161932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.161932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:51.162386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.162386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:51.162386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:51.162386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:51.162386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:51.162386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:51.162386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:51.162386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:51.162386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:51.162386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.162386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:51.162935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:51.163637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:51.164479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:51.165438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:51.165438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:51.165438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:51.165438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:51.165438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.165438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:51.165438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:51.165438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.165438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:51.165438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.165438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:51.172227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:51.172570 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:51.172685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:51.172856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:51.173232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:51.173446 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:51.173596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.174133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:51.174133 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.174133 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:51.174133 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:51.174133 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.174133 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:51.174133 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.174133 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:51.174997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.174997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:51.174997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:51.174997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:51.174997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:51.174997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:51.174997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:51.174997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.174997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:51.174997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:51.174997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:51.174997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.175517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:51.175517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:51.175517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:51.175517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:51.175517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:51.175517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:51.175517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:51.175517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:51.175517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.175517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:51.175517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:51.175517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.175517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:51.175889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.175889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:51.176003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:51.176824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:51.177791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:51.177791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:51.177791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:55.659406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:55.660316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:55.660572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:55.662056 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:55.700697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:55.700972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:55.701940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:55.701940 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:55.701940 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:55.701940 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:55.701940 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:55.701940 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:55.701940 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:55.701940 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:55.703138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:55.703138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:55.703138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:55.703138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:55.703138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:55.703138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:55.703138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:55.703138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:55.703138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:55.703138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:55.703138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:55.703138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:55.703732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:55.703732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:55.703732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:55.703732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:55.703732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:55.703732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:55.703732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:55.703732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:55.703732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:55.703732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:55.703732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:55.703732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:55.703732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:55.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:55.705006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:55.705794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:55.705794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:55.705794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:55.705794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:55.705794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:55.705794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:56.115276 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:56.116410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:56.117548 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:56.118784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:56.119107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:56.119385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.120680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:56.120680 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.120680 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:56.120680 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:56.120680 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.120680 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:56.120680 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.120680 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:56.122760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.122760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:56.122760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:56.122760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:56.122760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:56.122760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:56.122760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:56.122760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.122760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:56.122760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:56.122760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:56.122760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.123191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:56.123191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:56.123191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:56.123191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:56.123191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:56.123191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:56.123191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:56.123191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:56.123191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.123191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:56.123191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:56.123191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.123191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:56.123797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.123797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:56.123797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:56.124593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:56.125068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.125640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:56.125640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.125640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:56.125640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:56.125640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:56.125640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:33:56.125640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:56.125854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.126284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:56.126284 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.126284 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:56.126284 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:56.126284 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.126284 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:56.126284 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.126284 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:56.127030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:56.127576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:56.127576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:56.127576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.127576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:56.127576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:56.127576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.127576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:56.127576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.127576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:56.127874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.127874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:56.127874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:56.127874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:56.127874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:56.127874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:56.127874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:56.127874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:33:56.127874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:56.127874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:56.127874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.127874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:56.128259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:56.128696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.128696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:56.128696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.128696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:56.128942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:56.129425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:56.129999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.129999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:56.129999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:56.129999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.129999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:56.129999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.129999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:56.159102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:56.159609 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:56.159844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:56.160105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:56.160813 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:56.161124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.161855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:56.161855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.161855 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:56.161855 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:56.161855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.161855 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:56.161855 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.161855 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:33:56.162924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:33:56.163676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:56.163676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:33:56.163676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.163676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:33:56.163676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:33:56.163676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.163676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:33:56.163676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.163676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:33:56.164142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.164142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:56.164142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:56.164142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:56.164142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:56.164142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:56.164142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:56.164142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:56.164142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:56.164142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.164142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:56.164696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:56.165511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.165511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:56.165511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.165511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:56.165760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:56.166618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:56.167509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.167509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:56.167509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.167509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:33:56.172752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:33:56.172994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:33:56.173136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:33:56.173371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:33:56.174103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:56.174572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:33:56.174797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.175573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:33:56.175573 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.175573 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:33:56.175573 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:33:56.175573 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.175573 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:33:56.175573 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.175573 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:33:56.176560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.176560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:33:56.176560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:56.176560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:33:56.176560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:33:56.176560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:56.176560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:56.176560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.176560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:56.176560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:56.176560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:33:56.176560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.176970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:33:56.176970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:56.176970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:33:56.176970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:56.176970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:33:56.176970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:33:56.176970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:56.176970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:33:56.176970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.176970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:33:56.176970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:33:56.176970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.176970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:33:56.177536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.177536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:33:56.177849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:33:56.178579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:33:56.179406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:33:56.179406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:33:56.179406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:00.660063 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:00.660961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:00.660961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:00.662282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:00.695551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:00.696082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:00.696959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:00.696959 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:00.696959 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:00.696959 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:00.696959 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:00.696959 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:00.696959 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:00.696959 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:00.697798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:00.697798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:00.697798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:00.697798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:00.697798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:00.697798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:00.697798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:00.697798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:00.697798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:00.697798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:00.697798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:00.697798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:00.698424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:00.698424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:00.698424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:00.698424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:00.698424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:00.698424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:00.698424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:00.698424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:00.698424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:00.698424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:00.698424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:00.698424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:00.698424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:00.699026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:00.699854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:00.700736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:00.700736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:00.700736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:00.700736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:00.700736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:00.700736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:01.115815 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:01.116357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:01.116490 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:01.117505 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:01.117760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:01.117929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.118660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:01.118660 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.118660 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:01.118660 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:01.118660 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.118660 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:01.118660 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.118660 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:01.120094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.120094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:01.120094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:01.120094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:01.120094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:01.120094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:01.120094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:01.120094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.120094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:01.120094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:01.120094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:01.120094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:01.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:01.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:01.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:01.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:01.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:01.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:01.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:01.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:01.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:01.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:01.120895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.120895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:01.121804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:01.122359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.123011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:01.123011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.123011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:01.124993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:01.125408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:01.125685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:01.125951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:34:01.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:01.126694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.127291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:01.127291 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.127291 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:01.127291 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:01.127291 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.127291 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:01.127291 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.127291 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:01.128141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:01.128645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:01.128645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:01.128645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.128645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:01.128645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:01.128645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.128645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:01.128645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.128645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:01.129089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.129089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:01.129089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:01.129089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:01.129089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:01.129089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:01.129089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:01.129089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:34:01.129089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:01.129089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:01.129089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.129089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:01.129580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:01.130009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.130009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:01.130009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.130009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:01.130313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:01.130798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:01.131326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.131326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:01.131326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:01.131326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.131326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:01.131326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.131326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:01.159681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:01.160302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:01.160472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:01.160693 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:01.161222 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:01.161524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.162194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:01.162194 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.162194 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:01.162194 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:01.162194 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.162194 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:01.162194 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.162194 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:01.163445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:01.163868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:01.163868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:01.163868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.163868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:01.163868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:01.163868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.163868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:01.163868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.163868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:01.164763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.164763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:01.164763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:01.164763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:01.164763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:01.164763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:01.164763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:01.164763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:01.164763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:01.164763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.164763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:01.165124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:01.165507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.165507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:01.165507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.165507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:01.166376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:01.166969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:01.167439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.167439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:01.167439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.167439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:01.173242 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:01.173762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:01.173884 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:01.174096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:01.174622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:01.174996 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:01.175291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.176002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:01.176002 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.176002 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:01.176002 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:01.176002 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.176002 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:01.176002 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.176002 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:01.176884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.176884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:01.176884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:01.176884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:01.176884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:01.176884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:01.176884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:01.176884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.176884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:01.176884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:01.176884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:01.176884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.177331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:01.177331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:01.177331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:01.177331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:01.177331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:01.177331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:01.177331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:01.177331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:01.177331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.177331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:01.177331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:01.177331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.177331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:01.177708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.177708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:01.177924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:01.178442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:01.178951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:01.178951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:01.178951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:05.660777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:05.661392 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:05.661621 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:05.662758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:05.690505 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:05.691074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:05.691932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:05.691932 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:05.691932 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:05.691932 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:05.691932 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:05.691932 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:05.691932 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:05.691932 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:05.692697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:05.692697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:05.692697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:05.692697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:05.692697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:05.692697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:05.692697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:05.692697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:05.692697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:05.692697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:05.692697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:05.692697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:05.693232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:05.693232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:05.693232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:05.693232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:05.693232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:05.693232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:05.693232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:05.693232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:05.693232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:05.693232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:05.693232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:05.693232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:05.693232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:05.693676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:05.693676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:05.693760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:05.694308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:05.694919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:05.694919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:05.694919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:06.116441 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:06.116913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:06.117093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:06.118092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:06.118341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:06.118493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.119227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:06.119227 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.119227 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:06.119227 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:06.119227 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.119227 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:06.119227 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.119227 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:06.120620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.120620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:06.120620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:06.120620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:06.120620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:06.120620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:06.120620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:06.120620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.120620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:06.120620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:06.120620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:06.120620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.120904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:06.120904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:06.120904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:06.120904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:06.120904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:06.120904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:06.120904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:06.120904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:06.120904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.120904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:06.120904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:06.120904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.120904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:06.121135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.121135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:06.122224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:06.122611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.123002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:06.123002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.123002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:06.125660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:06.125660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:06.125747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:06.125854 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:34:06.126300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:06.126678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.127312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:06.127312 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.127312 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:06.127312 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:06.127312 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.127312 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:06.127312 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.127312 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:06.128090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:06.128466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:06.128466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:06.128466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.128466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:06.128466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:06.128466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.128466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:06.128466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.128466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:06.128995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.128995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:06.128995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:06.128995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:06.128995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:06.128995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:06.128995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:06.128995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:34:06.128995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:06.128995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:06.128995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.128995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:06.129627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:06.130141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:06.131112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:06.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:06.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:06.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:06.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:06.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:06.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:06.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:06.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:06.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:06.160461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:06.160825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:06.160949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:06.161019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:06.161801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:06.162096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.162677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:06.162677 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.162677 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:06.162677 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:06.162677 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.162677 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:06.162677 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.162677 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:06.163649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:06.164343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:06.164343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:06.164343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.164343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:06.164343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:06.164343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.164343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:06.164343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.164343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:06.164840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.164840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:06.164840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:06.164840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:06.164840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:06.164840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:06.164840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:06.164840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:06.164840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:06.164840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.164840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:06.165374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:06.165954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:06.168722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:06.170791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:06.170791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:06.170791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:06.170791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:06.170791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.170791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:06.170791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:06.170791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.170791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:06.170791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.170791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:06.173800 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:06.174095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:06.174195 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:06.174434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:06.175015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:06.175654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:06.175834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.176477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:06.176477 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.176477 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:06.176477 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:06.176477 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.176477 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:06.176477 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.176477 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:06.177803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.177803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:06.177803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:06.177803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:06.177803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:06.177803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:06.177803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:06.177803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.177803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:06.177803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:06.177803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:06.177803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.178087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:06.178087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:06.178087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:06.178087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:06.178087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:06.178087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:06.178087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:06.178087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:06.178087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.178087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:06.178087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:06.178087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.178087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:06.178336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.178336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:06.179184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:06.179729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:06.180094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:06.180094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:06.180094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:10.661795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:10.662715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:10.662715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:10.663694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:10.724169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:10.724470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:10.725146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:10.725146 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:10.725146 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:10.725146 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:10.725146 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:10.725146 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:10.725146 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:10.725146 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:10.725818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:10.725818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:10.725818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:10.725818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:10.725818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:10.725818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:10.725818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:10.725818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:10.725818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:10.725818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:10.725818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:10.725818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:10.726489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:10.726489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:10.726489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:10.726489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:10.726489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:10.726489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:10.726489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:10.726489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:10.726489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:10.726489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:10.726489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:10.726489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:10.726489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:10.727122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:10.727959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:10.728866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:10.728866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:10.728866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:10.728866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:10.728866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:10.728866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:11.117051 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:11.117617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:11.117812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:11.118912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:11.119448 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:11.119698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.120686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:11.120686 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.120686 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:11.120686 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:11.120686 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.120686 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:11.120686 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.120686 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:11.122128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.122128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:11.122128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:11.122128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:11.122128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:11.122128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:11.122128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:11.122128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.122128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:11.122128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:11.122128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:11.122128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.122935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:11.122935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:11.122935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:11.122935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:11.122935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:11.122935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:11.122935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:11.122935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:11.122935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.122935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:11.122935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:11.122935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.122935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:11.123548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.123548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:11.123773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:11.124673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.125751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:11.125751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.125751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:11.125751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:11.126094 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:11.126278 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:11.126479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:34:11.126842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:11.127089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.127713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:11.127713 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.127713 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:11.127713 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:11.127713 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.127713 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:11.127713 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.127713 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:11.128875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:11.129441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:11.129441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:11.129441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.129441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:11.129441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:11.129441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.129441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:11.129441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.129441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:11.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:11.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:11.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:11.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:11.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:11.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:11.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:34:11.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:11.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:11.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:11.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:11.131021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.131021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:11.131021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.131021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:11.131743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:11.132172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:11.132863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.132863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:11.132863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:11.132863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.132863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:11.132863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.132863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:11.161101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:11.161519 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:11.161681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:11.161879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:11.162600 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:11.163003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.163855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:11.163855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.163855 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:11.163855 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:11.163855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.163855 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:11.163855 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.163855 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:11.165060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:11.165907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:11.165907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:11.165907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.165907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:11.165907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:11.165907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.165907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:11.165907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.165907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:11.166430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.166430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:11.166430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:11.166430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:11.166430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:11.166430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:11.166430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:11.166430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:11.166430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:11.166430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.166430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:11.166944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:11.167612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.167612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:11.167612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.167612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:11.168097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:11.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:11.169866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.169866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:11.169866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.169866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:11.174279 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:11.174710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:11.174961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:11.175345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:11.175823 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:11.176066 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:11.176240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.176786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:11.176786 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.176786 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:11.176786 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:11.176786 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.176786 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:11.176786 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.176786 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:11.177682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.177682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:11.177682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:11.177682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:11.177682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:11.177682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:11.177682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:11.177682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.177682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:11.177682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:11.177682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:11.177682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.178034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:11.178034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:11.178034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:11.178034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:11.178034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:11.178034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:11.178034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:11.178034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:11.178034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.178034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:11.178034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:11.178034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.178034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:11.178440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.178440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:11.178655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:11.179084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:11.179815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:11.179815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:11.179815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:15.662445 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:15.663383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:15.663383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:15.664313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:15.696590 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:15.696971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:15.697950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:15.697950 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:15.697950 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:15.697950 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:15.697950 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:15.697950 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:15.697950 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:15.697950 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:15.698900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:15.698900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:15.698900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:15.698900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:15.698900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:15.698900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:15.698900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:15.698900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:15.698900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:15.698900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:15.698900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:15.698900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:15.699553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:15.699553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:15.699553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:15.699553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:15.699553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:15.699553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:15.699553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:15.699553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:15.699553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:15.699553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:15.699553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:15.699553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:15.699553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:15.700204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:15.701168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:15.702472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:15.702472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:15.702472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:15.702472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:15.702472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:15.702472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:16.117718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:16.118145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:16.118479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:16.119514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:16.119831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:16.120054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.120884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:16.120884 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.120884 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:16.120884 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:16.120884 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.120884 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:16.120884 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.120884 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:16.122388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.122388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:16.122388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:16.122388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:16.122388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:16.122388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:16.122388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:16.122388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.122388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:16.122388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:16.122388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:16.122388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:16.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:16.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:16.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:16.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:16.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:16.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:16.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:16.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:16.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:16.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.122797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:16.123178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.123178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:16.124087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:16.124585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:16.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:16.126179 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:16.126561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:16.126674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:16.126781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:34:16.127217 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:16.127538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.128163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:16.128163 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.128163 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:16.128163 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:16.128163 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.128163 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:16.128163 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.128163 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:16.128926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:16.129442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:16.129442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:16.129442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.129442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:16.129442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:16.129442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.129442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:16.129442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.129442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:16.129782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.129782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:16.129782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:16.129782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:16.129782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:16.129782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:16.129782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:16.129782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:34:16.129782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:16.129782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:16.129782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.129782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:16.130218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:16.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:16.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.130644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:16.130864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:16.131435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:16.131979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.131979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:16.131979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:16.131979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.131979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:16.131979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.131979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:16.161719 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:16.162137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:16.162329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:16.162539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:16.163017 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:16.163413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.164160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:16.164160 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.164160 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:16.164160 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:16.164160 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.164160 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:16.164160 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.164160 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:16.165398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:16.165831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:16.165831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:16.165831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.165831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:16.165831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:16.165831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.165831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:16.165831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.165831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:16.166875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.166875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:16.166875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:16.166875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:16.166875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:16.166875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:16.166875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:16.166875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:16.166875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:16.166875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.166875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:16.167242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:16.167619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.167619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:16.167619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.167619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:16.168346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:16.168838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:16.169617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.169617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:16.169617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.169617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:16.174707 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:16.174999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:16.175113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:16.175319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:16.175756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:16.176097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:16.176358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.177169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:16.177169 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.177169 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:16.177169 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:16.177169 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.177169 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:16.177169 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.177169 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:16.178749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.178749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:16.178749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:16.178749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:16.178749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:16.178749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:16.178749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:16.178749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.178749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:16.178749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:16.178749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:16.178749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.179185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:16.179185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:16.179185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:16.179185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:16.179185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:16.179185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:16.179185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:16.179185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:16.179185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.179185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:16.179185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:16.179185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.179185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:16.179631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.179631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:16.180528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:16.181070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:16.181649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:16.181649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:16.181649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:20.662988 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:20.663900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:20.663900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:20.664829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:20.697707 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:20.697982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:20.698628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:20.698628 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:20.698628 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:20.698628 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:20.698628 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:20.698628 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:20.698628 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:20.698628 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:20.699551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:20.699551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:20.699551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:20.699551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:20.699551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:20.699551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:20.699551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:20.699551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:20.699551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:20.699551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:20.699551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:20.699551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:20.700244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:20.700244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:20.700244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:20.700244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:20.700244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:20.700244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:20.700244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:20.700244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:20.700244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:20.700244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:20.700244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:20.700244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:20.700244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:20.701091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:20.701785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:20.702764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:20.702764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:20.702764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:20.702764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:20.702764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:20.702764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:21.118270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:21.118798 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:21.118925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:21.119760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:21.120078 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:21.120388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.121309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:21.121309 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.121309 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:21.121309 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:21.121309 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.121309 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:21.121309 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.121309 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:21.122972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.122972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:21.122972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:21.122972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:21.122972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:21.122972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:21.122972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:21.122972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.122972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:21.122972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:21.122972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:21.122972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:21.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:21.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:21.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:21.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:21.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:21.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:21.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:21.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:21.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:21.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.123785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:21.124373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:21.125315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.126511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:21.126511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:21.126511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.126511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:21.126511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.126511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:21.127029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:21.127029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:21.127140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:21.127198 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:34:21.127539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:21.127722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.128469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:21.128469 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.128469 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:21.128469 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:21.128469 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.128469 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:21.128469 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.128469 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:21.129182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:21.129853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:21.129853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:21.129853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.129853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:21.129853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:21.129853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.129853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:21.129853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.129853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:21.130338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.130338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:21.130338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:21.130338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:21.130338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:21.130338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:21.130338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:21.130338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:34:21.130338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:21.130338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:21.130338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.130338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:21.130900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:21.131587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:21.132611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:21.133581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:21.133581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:21.133581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:21.133581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:21.133581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:21.133581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.133581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:21.133581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:21.133581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.133581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:21.133581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.133581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:21.162257 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:21.162702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:21.162887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:21.163091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:21.163563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:21.163939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.164741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:21.164741 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.164741 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:21.164741 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:21.164741 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.164741 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:21.164741 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.164741 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:21.165881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:21.166608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:21.166608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:21.166608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.166608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:21.166608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:21.166608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.166608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:21.166608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.166608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:21.167079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.167079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:21.167079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:21.167079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:21.167079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:21.167079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:21.167079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:21.167079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:21.167079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:21.167079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.167079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:21.167693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:21.168329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:21.168948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:21.169915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:21.169915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:21.169915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:21.169915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:21.169915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.169915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:21.169915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:21.169915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.169915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:21.169915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.169915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:21.175179 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:21.175460 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:21.175555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:21.175712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:21.175944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:21.176165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:21.176356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.176815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:21.176815 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.176815 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:21.176815 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:21.176815 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.176815 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:21.176815 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.176815 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:21.177687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.177687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:21.177687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:21.177687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:21.177687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:21.177687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:21.177687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:21.177687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.177687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:21.177687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:21.177687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:21.177687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.178069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:21.178069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:21.178069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:21.178069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:21.178069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:21.178069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:21.178069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:21.178069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:21.178069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.178069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:21.178069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:21.178069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.178069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:21.178454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.178454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:21.178827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:21.179308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:21.179855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:21.179855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:21.179855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:25.663490 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:25.663962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:25.664091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:25.664967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:25.700389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:25.700604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:25.701077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:25.701077 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:25.701077 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:25.701077 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:25.701077 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:25.701077 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:25.701077 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:25.701077 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:25.701950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:25.701950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:25.701950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:25.701950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:25.701950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:25.701950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:25.701950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:25.701950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:25.701950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:25.701950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:25.701950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:25.701950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:25.702354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:25.702354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:25.702354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:25.702354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:25.702354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:25.702354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:25.702354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:25.702354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:25.702354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:25.702354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:25.702354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:25.702354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:25.702354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:25.702751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:25.702751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:25.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:25.703465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:25.703961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:25.703961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:25.703961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:26.118916 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:26.119658 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:26.119953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:26.120956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:26.121307 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:26.121605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.122508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:26.122508 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.122508 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:26.122508 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:26.122508 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.122508 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:26.122508 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.122508 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:26.123970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.123970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:26.123970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:26.123970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:26.123970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:26.123970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:26.123970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:26.123970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.123970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:26.123970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:26.123970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:26.123970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.124581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:26.124581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:26.124581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:26.124581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:26.124581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:26.124581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:26.124581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:26.124581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:26.124581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.124581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:26.124581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:26.124581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.124581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:26.125130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.125130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:26.125465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:26.126173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.127047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:26.127047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.127047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:26.127322 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:26.127547 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:26.127637 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:26.127886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:34:26.128185 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:26.128422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.128889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:26.128889 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.128889 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:26.128889 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:26.128889 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.128889 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:26.128889 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.128889 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:26.129730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:26.130407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:26.130407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:26.130407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.130407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:26.130407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:26.130407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.130407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:26.130407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.130407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:26.130817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.130817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:26.130817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:26.130817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:26.130817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:26.130817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:26.130817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:26.130817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:34:26.130817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:26.130817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:26.130817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.130817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:26.131240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:26.131680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.131680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:26.131680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.131680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:26.132095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:26.132581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:26.133330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.133330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:26.133330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:26.133330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.133330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:26.133330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.133330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:26.162849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:26.163258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:26.163569 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:26.163809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:26.164505 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:26.164792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.165475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:26.165475 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.165475 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:26.165475 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:26.165475 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.165475 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:26.165475 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.165475 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:26.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:26.167518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:26.167518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:26.167518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.167518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:26.167518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:26.167518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.167518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:26.167518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.167518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:26.168676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.168676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:26.168676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:26.168676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:26.168676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:26.168676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:26.168676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:26.168676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:26.168676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:26.168676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.168676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:26.169280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:26.169899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.169899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:26.169899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.169899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:26.170794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:26.171727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:26.172309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.172309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:26.172309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.172309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:26.175675 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:26.175957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:26.176087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:26.176297 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:26.176794 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:26.177172 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:26.177525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.178147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:26.178147 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.178147 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:26.178147 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:26.178147 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.178147 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:26.178147 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.178147 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:26.179023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.179023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:26.179023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:26.179023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:26.179023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:26.179023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:26.179023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:26.179023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.179023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:26.179023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:26.179023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:26.179023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.179539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:26.179539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:26.179539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:26.179539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:26.179539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:26.179539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:26.179539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:26.179539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:26.179539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.179539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:26.179539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:26.179539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.179539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:26.179913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.179913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:26.180344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:26.180821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:26.181381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:26.181381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:26.181381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:30.664253 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:30.664999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:30.664999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:30.665638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:30.699537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:30.700893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:30.700893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:30.700893 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:30.700893 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:30.700893 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:30.700893 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:30.700893 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:30.700893 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:30.700893 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:30.702070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:30.702070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:30.702070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:30.702070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:30.702070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:30.702070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:30.702070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:30.702070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:30.702070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:30.702070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:30.702070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:30.702070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:30.702773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:30.702773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:30.702773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:30.702773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:30.702773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:30.702773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:30.702773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:30.702773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:30.702773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:30.702773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:30.702773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:30.702773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:30.702773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:30.703187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:30.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:30.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:30.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:30.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:30.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:30.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:30.704307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:31.119525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:31.122496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:31.122650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:31.123419 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:31.123751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:31.124096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.124885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:31.124885 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.124885 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:31.124885 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:31.124885 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.124885 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:31.124885 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.124885 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:31.125680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.125680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:31.125680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:31.125680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:31.125680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:31.125680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:31.125680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:31.125680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.125680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:31.125680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:31.125680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:31.125680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.126149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:31.126149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:31.126149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:31.126149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:31.126149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:31.126149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:31.126149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:31.126149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:31.126149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.126149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:31.126149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:31.126149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.126149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:31.126539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.126539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:31.126732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:31.127345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.128021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:31.128021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.128021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:31.128021 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:31.128167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:31.128167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:31.128292 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:34:31.128762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:31.129104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.129773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:31.129773 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.129773 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:31.129773 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:31.129773 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.129773 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:31.129773 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.129773 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:31.130344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:31.130882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:31.131300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:31.131300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:31.131300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:34:31.131300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:31.131300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:31.131300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.131300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:31.131300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:31.131300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:31.131300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.131300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:31.131300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:31.131300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:31.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:31.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:31.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:31.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:31.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:31.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:31.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:31.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:31.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.131701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:31.132021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:31.132609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:31.133425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.133425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:31.133425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:31.133425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.133425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:31.133425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.133425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:31.163247 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:31.163620 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:31.163792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:31.163998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:31.164611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:31.164992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.165828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:31.165828 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.165828 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:31.165828 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:31.165828 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.165828 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:31.165828 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.165828 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:31.167073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:31.167984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:31.167984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:31.167984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.167984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:31.167984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:31.167984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.167984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:31.167984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.167984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:31.168690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.168690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:31.168690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:31.168690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:31.168690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:31.168690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:31.168690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:31.168690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:31.168690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:31.168690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.168690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:31.169310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:31.169983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.169983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:31.169983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.169983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:31.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:31.171367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:31.172234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.172234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:31.172234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.172234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:31.176662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:31.176996 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:31.177137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:31.177322 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:31.178007 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:31.179291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:31.179731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.180567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:31.180567 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.180567 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:31.180567 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:31.180567 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.180567 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:31.180567 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.180567 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:31.181978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.181978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:31.181978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:31.181978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:31.181978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:31.181978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:31.181978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:31.181978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.181978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:31.181978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:31.181978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:31.181978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.182651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:31.182651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:31.182651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:31.182651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:31.182651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:31.182651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:31.182651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:31.182651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:31.182651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.182651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:31.182651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:31.182651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.182651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:31.183233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.183233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:31.183664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:31.184385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:31.185187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:31.185187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:31.185187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:35.664702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:35.665495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:35.665495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:35.666720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:35.699071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:35.699480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:35.700284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:35.700284 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:35.700284 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:35.700284 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:35.700284 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:35.700284 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:35.700284 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:35.700284 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:35.701053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:35.701053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:35.701053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:35.701053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:35.701053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:35.701053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:35.701053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:35.701053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:35.701053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:35.701053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:35.701053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:35.701053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:35.701965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:35.701965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:35.701965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:35.701965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:35.701965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:35.701965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:35.701965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:35.701965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:35.701965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:35.701965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:35.701965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:35.701965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:35.701965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:35.702856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:35.703964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:35.705271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:35.705271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:35.705271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:35.705271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:35.705271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:35.705271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:36.122708 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:36.123305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:36.123513 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:36.124681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:36.124972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:36.125190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.126047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:36.126047 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.126047 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:36.126047 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:36.126047 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.126047 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:36.126047 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.126047 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:36.127714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.127714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:36.127714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:36.127714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:36.127714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:36.127714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:36.127714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:36.127714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.127714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:36.127714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:36.127714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:36.127714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.128098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:36.128098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:36.128098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:36.128098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:36.128098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:36.128098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:36.128098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:36.128098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:36.128098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.128098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:36.128098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:36.128098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.128098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:36.128500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.128500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:36.129303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:36.129746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:36.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:36.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:36.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:36.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:36.130270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:34:36.130487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:36.130726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.131390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:36.131390 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.131390 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:36.131390 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:36.131390 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.131390 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:36.131390 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.131390 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:36.132094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:36.132531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:36.132531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:36.132531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.132531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:36.132531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:36.132531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.132531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:36.132531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.132531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:36.132935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.132935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:36.132935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:36.132935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:36.132935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:36.132935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:36.132935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:36.132935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:34:36.132935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:36.132935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:36.132935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.132935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:36.133310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:36.133689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.133689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:36.133689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.133689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:36.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:36.134447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:36.134927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.134927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:36.134927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:36.134927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.134927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:36.134927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.134927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:36.163834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:36.164282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:36.164476 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:36.164665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:36.165156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:36.165552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.166202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:36.166202 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.166202 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:36.166202 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:36.166202 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.166202 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:36.166202 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.166202 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:36.167496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:36.167920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:36.167920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:36.167920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.167920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:36.167920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:36.167920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.167920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:36.167920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.167920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:36.168824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.168824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:36.168824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:36.168824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:36.168824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:36.168824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:36.168824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:36.168824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:36.168824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:36.168824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.168824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:36.169162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:36.169548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.169548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:36.169548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.169548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:36.170495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:36.170963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:36.171496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.171496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:36.171496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.171496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:36.177294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:36.177610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:36.177721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:36.177873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:36.178255 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:36.178572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:36.178783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.179505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:36.179505 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.179505 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:36.179505 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:36.179505 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.179505 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:36.179505 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.179505 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:36.180303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.180303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:36.180303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:36.180303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:36.180303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:36.180303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:36.180303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:36.180303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.180303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:36.180303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:36.180303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:36.180303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.180650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:36.180650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:36.180650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:36.180650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:36.180650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:36.180650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:36.180650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:36.180650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:36.180650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.180650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:36.180650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:36.180650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.180650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:36.180983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.180983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:36.181273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:36.181700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:36.182198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:36.182198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:36.182198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:40.665404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:40.666167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:40.666167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:40.667442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:40.674641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=241 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:34:40.704927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:40.705399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:40.706321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:40.706321 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:40.706321 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:40.706321 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:40.706321 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:40.706321 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:40.706321 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:40.706321 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:40.707267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:40.707267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:40.707267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:40.707267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:40.707267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:40.707267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:40.707267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:40.707267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:40.707267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:40.707267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:40.707267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:40.707267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:40.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:40.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:40.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:40.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:40.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:40.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:40.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:40.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:40.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:40.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:40.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:40.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:40.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:40.708490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:40.709147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:40.710001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:40.710001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:40.710001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:40.710001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:40.710001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:40.710001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:41.123331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:41.123965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:41.124166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:41.125360 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:41.125703 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:41.125914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.126762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:41.126762 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.126762 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:41.126762 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:41.126762 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.126762 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:41.126762 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.126762 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:41.128333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.128333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:41.128333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:41.128333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:41.128333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:41.128333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:41.128333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:41.128333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.128333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:41.128333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:41.128333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:41.128333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.128919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:41.128919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:41.128919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:41.128919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:41.128919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:41.128919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:41.128919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:41.128919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:41.128919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.128919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:41.128919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:41.128919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.128919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:41.129444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.129444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:41.129973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:41.131630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.133167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:41.133167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.133167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:41.133167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:41.133167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:41.133167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:41.133167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:34:41.133167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:41.134020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.135274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:41.136099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.136099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:41.136099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:41.136099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:41.136099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:41.136099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:41.136099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:41.136099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:34:41.136099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:41.136099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:41.136099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.136099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:41.138223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:41.138485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:41.138794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:41.139177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:41.139177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:41.139177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:41.139177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:41.139177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:41.139177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.139177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:41.139177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:41.139177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.139177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:41.139177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.139177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:41.164415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:41.164860 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:41.165049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:41.165253 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:41.166157 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:41.166663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.167721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:41.167721 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.167721 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:41.167721 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:41.167721 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.167721 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:41.167721 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.167721 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:41.169761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:41.170550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:41.170550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:41.170550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.170550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:41.170550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:41.170550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.170550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:41.170550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.170550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:41.172039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.172039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:41.172039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:41.172039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:41.172039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:41.172039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:41.172039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:41.172039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:41.172039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:41.172039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.172039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:41.172608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:41.173200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.173200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:41.173200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.173200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:41.173714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:41.174505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:41.175363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.175363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:41.175363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.175363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:41.177755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:41.178054 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:41.178410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:41.178786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:41.179364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:41.179726 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:41.179969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.180738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:41.180738 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.180738 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:41.180738 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:41.180738 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.180738 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:41.180738 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.180738 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:41.181623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.181623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:41.181623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:41.181623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:41.181623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:41.181623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:41.181623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:41.181623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.181623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:41.181623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:41.181623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:41.181623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.182011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:41.182011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:41.182011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:41.182011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:41.182011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:41.182011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:41.182011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:41.182011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:41.182011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.182011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:41.182011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:41.182011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.182011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:41.182421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.182421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:41.182700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:41.183148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:41.183702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:41.183702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:41.183702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:45.665922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:45.666768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:45.666768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:45.667846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:45.696592 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:45.696998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:45.697963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:45.697963 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:45.697963 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:45.697963 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:45.697963 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:45.697963 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:45.697963 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:45.697963 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:45.698950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:45.698950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:45.698950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:45.698950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:45.698950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:45.698950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:45.698950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:45.698950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:45.698950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:45.698950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:45.698950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:45.698950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:45.699641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:45.699641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:45.699641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:45.699641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:45.699641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:45.699641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:45.699641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:45.699641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:45.699641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:45.699641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:45.699641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:45.699641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:45.699641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:45.700152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:45.700152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:45.700272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:45.700875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:45.701654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:45.701654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:45.701654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:46.124093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:46.124697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:46.124999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:46.126131 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:46.126372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:46.126496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.127106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:46.127106 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.127106 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:46.127106 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:46.127106 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.127106 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:46.127106 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.127106 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:46.129097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.129097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:46.129097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:46.129097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:46.129097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:46.129097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:46.129097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:46.129097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.129097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:46.129097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:46.129097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:46.129097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:46.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:46.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:46.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:46.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:46.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:46.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:46.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:46.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:46.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:46.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:46.130293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.130293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:46.131195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:46.131939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.132770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:46.132770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.132770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:46.132770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:46.132770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:46.132770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:46.132770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:34:46.133096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:46.133586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.134456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:46.134456 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.134456 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:46.134456 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:46.134456 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.134456 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:46.134456 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.134456 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:46.135398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:46.136199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:46.137033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:46.137033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:46.137033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:34:46.137033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:46.137033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:46.137033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.137033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:46.137033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:46.137033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:46.137033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.137033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:46.137033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:46.137033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:46.137853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:46.137853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:46.137853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:46.137853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:46.137853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:46.137853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.137853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:46.137853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:46.137853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.137853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:46.137853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.137853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:46.138428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:46.138824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:46.139868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.139868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:46.139868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:46.139868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.139868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:46.139868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.139868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:46.164956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:46.165364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:46.165475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:46.165626 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:46.166028 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:46.166311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.166862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:46.166862 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.166862 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:46.166862 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:46.166862 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.166862 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:46.166862 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.166862 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:46.167650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:46.168073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:46.168073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:46.168073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.168073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:46.168073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:46.168073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.168073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:46.168073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.168073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:46.168706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.168706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:46.168706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:46.168706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:46.168706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:46.168706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:46.168706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:46.168706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:46.168706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:46.168706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.168706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:46.169386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:46.169855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:46.170240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:46.170723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:46.170723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:46.170723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:46.170723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:46.170723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.170723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:46.170723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:46.170723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.170723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:46.170723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.170723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:46.178591 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:46.178830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:46.178957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:46.179087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:46.179438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:46.179686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:46.179873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.180518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:46.180518 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.180518 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:46.180518 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:46.180518 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.180518 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:46.180518 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.180518 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:46.181523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.181523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:46.181523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:46.181523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:46.181523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:46.181523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:46.181523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:46.181523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.181523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:46.181523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:46.181523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:46.181523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:46.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:46.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:46.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:46.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:46.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:46.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:46.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:46.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:46.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:46.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:46.182656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:46.183264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:46.183980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:46.183980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:46.183980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:46.183980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:46.183980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:46.183980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:50.666832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:50.667863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:50.667863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:50.668876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:50.703926 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:50.704139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:50.704676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:50.704676 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:50.704676 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:50.704676 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:50.704676 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:50.704676 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:50.704676 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:50.704676 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:50.705426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:50.705426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:50.705426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:50.705426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:50.705426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:50.705426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:50.705426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:50.705426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:50.705426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:50.705426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:50.705426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:50.705426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:50.706133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:50.706133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:50.706133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:50.706133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:50.706133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:50.706133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:50.706133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:50.706133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:50.706133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:50.706133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:50.706133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:50.706133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:50.706133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:50.706809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:50.707529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:50.709662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:50.709662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:50.709662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:50.709662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:50.709662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:50.709662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:51.124645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:51.124840 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:51.124947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:51.126108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:51.126108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:51.126108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.126773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:51.126773 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.126773 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:51.126773 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:51.126773 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.126773 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:51.126773 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.126773 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:51.127787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.127787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:51.127787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:51.127787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:51.127787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:51.127787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:51.127787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:51.127787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.127787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:51.127787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:51.127787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:51.127787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.128282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:51.128282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:51.128282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:51.128282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:51.128282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:51.128282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:51.128282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:51.128282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:51.128282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.128282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:51.128282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:51.128282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.128282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:51.128627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.128627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:51.128844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:51.129809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.130305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:51.130305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.130305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:51.131922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:51.132121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:51.132121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:51.132329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:34:51.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:51.132969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.133604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:51.133604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.133604 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:51.133604 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:51.133604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.133604 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:51.133604 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.133604 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:51.134326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:51.134818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:51.134818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:51.134818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.134818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:51.134818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:51.134818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.134818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:51.134818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.134818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:51.135124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.135124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:51.135124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:51.135124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:51.135124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:51.135124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:51.135124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:51.135124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:34:51.135124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:51.135124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:51.135124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.135124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:51.135509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:51.135906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.135906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:51.135906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.135906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:51.136086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:51.136571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:51.137066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.137066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:51.137066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:51.137066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.137066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:51.137066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.137066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:51.165524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:51.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:51.165730 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:51.165886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:51.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:51.166835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.167505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:51.167505 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.167505 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:51.167505 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:51.167505 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.167505 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:51.167505 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.167505 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:51.168294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:51.169003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:51.169003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:51.169003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.169003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:51.169003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:51.169003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.169003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:51.169003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.169003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:51.169477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.169477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:51.169477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:51.169477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:51.169477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:51.169477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:51.169477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:51.169477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:51.169477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:51.169477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.169477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:51.170019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:51.170662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:51.171355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:51.171928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:51.171928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:51.171928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:51.171928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:51.171928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.171928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:51.171928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:51.171928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.171928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:51.171928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.171928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:51.178830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:51.179006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:51.179113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:51.179260 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:51.179730 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:51.180073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:51.180288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.180814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:51.180814 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.180814 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:51.180814 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:51.180814 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.180814 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:51.180814 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.180814 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:51.181629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.181629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:51.181629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:51.181629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:51.181629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:51.181629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:51.181629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:51.181629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.181629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:51.181629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:51.181629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:51.181629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.182200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:51.182200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:51.182200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:51.182200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:51.182200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:51.182200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:51.182200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:51.182200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:51.182200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.182200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:51.182200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:51.182200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.182200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:51.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:51.183481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:51.184282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:51.184282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:51.184282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:51.184282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:51.184282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:51.184282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:55.667652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:55.668448 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:55.668448 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:55.669695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:55.702930 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:55.703330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:55.704234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:55.704234 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:55.704234 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:55.704234 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:55.704234 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:55.704234 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:55.704234 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:55.704234 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:55.705171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:55.705171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:55.705171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:55.705171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:55.705171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:55.705171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:55.705171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:55.705171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:55.705171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:55.705171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:55.705171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:55.705171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:55.705764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:55.705764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:55.705764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:55.705764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:55.705764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:55.705764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:55.705764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:55.705764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:55.705764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:55.705764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:55.705764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:55.705764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:55.705764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:55.706322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:55.707006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:55.707816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:55.707816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:55.707816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:55.707816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:55.707816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:55.707816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:56.125200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:56.125721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:56.125835 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:56.126982 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:56.127298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:56.127540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.128494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:56.128494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.128494 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:56.128494 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:56.128494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.128494 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:56.128494 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.128494 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:56.130114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.130114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:56.130114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:56.130114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:56.130114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:56.130114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:56.130114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:56.130114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.130114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:56.130114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:56.130114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:56.130114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.130682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:56.130682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:56.130682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:56.130682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:56.130682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:56.130682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:56.130682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:56.130682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:56.130682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.130682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:56.130682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:56.130682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.130682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:56.131267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.131267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:56.131870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:56.132579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.133387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:56.133387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.133387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:56.133387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:56.133387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:56.133387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:56.133387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:34:56.133387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:56.133766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.134309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:56.134309 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.134309 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:56.134309 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:56.134309 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.134309 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:56.134309 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.134309 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:56.135077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:56.135819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:56.135819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:56.135819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.135819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:56.135819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:56.135819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.135819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:56.135819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.135819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:56.136236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.136236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:56.136236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:56.136236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:56.136236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:56.136236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:56.136236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:56.136236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:34:56.136236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:56.136236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:56.136236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.136236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:56.136737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:56.137341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:56.137915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:56.138714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:56.138714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:56.138714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:56.138714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:56.138714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:56.138714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.138714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:56.138714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:56.138714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.138714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:56.138714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.138714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:56.165836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:56.166242 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:56.166483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:56.166694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:56.167197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:56.167597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.168435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:56.168435 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.168435 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:56.168435 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:56.168435 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.168435 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:56.168435 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.168435 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:34:56.169664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:34:56.170295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:56.170295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:34:56.170295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.170295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:34:56.170295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:34:56.170295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.170295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:34:56.170295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.170295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:34:56.171049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.171049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:56.171049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:56.171049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:56.171049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:56.171049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:56.171049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:56.171049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:56.171049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:56.171049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.171049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:56.171529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:56.172166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.172166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:56.172166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.172166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:56.172886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:56.173665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:56.174459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.174459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:56.174459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.174459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:34:56.179153 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:34:56.179510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:34:56.179650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:34:56.179851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:34:56.180254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:56.180580 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:34:56.180792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.181463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:34:56.181463 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.181463 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:34:56.181463 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:34:56.181463 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.181463 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:34:56.181463 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.181463 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:34:56.182356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.182356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:34:56.182356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:56.182356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:34:56.182356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:34:56.182356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:56.182356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:56.182356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.182356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:56.182356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:56.182356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:34:56.182356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.182775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:34:56.182775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:56.182775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:34:56.182775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:56.182775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:34:56.182775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:34:56.182775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:56.182775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:34:56.182775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.182775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:34:56.182775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:34:56.182775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.182775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:34:56.183121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.183121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:34:56.183385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:34:56.183877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:34:56.184415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:34:56.184415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:34:56.184415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:00.668409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:00.669477 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:00.669477 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:00.670080 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:00.706659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:00.706659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:00.707478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:00.707478 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:00.707478 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:00.707478 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:00.707478 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:00.707478 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:00.707478 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:00.707478 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:00.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:00.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:00.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:00.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:00.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:00.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:00.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:00.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:00.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:00.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:00.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:00.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:00.709307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:00.709307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:00.709307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:00.709307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:00.709307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:00.709307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:00.709307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:00.709307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:00.709307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:00.709307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:00.709307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:00.709307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:00.709307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:00.709909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:00.710870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:00.711706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:00.711706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:00.711706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:00.711706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:00.711706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:00.711706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:01.125837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:01.126273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:01.126444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:01.127566 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:01.128119 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:01.128646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.129361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:01.129361 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.129361 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:01.129361 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:01.129361 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.129361 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:01.129361 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.129361 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:01.130592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.130592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:01.130592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:01.130592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:01.130592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:01.130592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:01.130592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:01.130592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.130592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:01.130592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:01.130592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:01.130592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.131353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:01.131353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:01.131353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:01.131353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:01.131353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:01.131353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:01.131353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:01.131353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:01.131353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.131353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:01.131353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:01.131353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.131353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:01.131985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:01.132775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.134134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:01.134134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:01.134134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.134134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:01.134134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.134134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:01.134134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:01.134134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:01.134134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:01.134134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:35:01.134862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:01.135160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.135767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:01.135767 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.135767 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:01.135767 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:01.135767 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.135767 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:01.135767 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.135767 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:01.136535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:01.137349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:01.137868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:01.137868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:01.137868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:35:01.137868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:01.137868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:01.137868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.137868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:01.137868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:01.137868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:01.137868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.137868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:01.137868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:01.137868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:01.138268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:01.138268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:01.138268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:01.138268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:01.138268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:01.138268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.138268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:01.138268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:01.138268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.138268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:01.138268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.138268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:01.139028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:01.139566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:01.140184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.140184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:01.140184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:01.140184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.140184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:01.140184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.140184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:01.166332 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:01.166696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:01.166869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:01.167060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:01.167707 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:01.168122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.168980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:01.168980 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.168980 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:01.168980 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:01.168980 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.168980 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:01.168980 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.168980 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:01.169988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:01.170641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:01.171291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:01.171291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:01.171291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:01.171291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:01.171291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.171291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:01.171291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:01.171291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:01.171291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.171291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:01.171291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:01.171291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:01.171291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:01.171885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:01.171885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:01.171885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:01.171885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:01.171885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.171885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:01.171885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:01.171885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.171885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:01.171885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.171885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:01.171885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:01.172453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.173330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:01.179526 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:01.179763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:01.179863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:01.180014 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:01.180535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:01.181002 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:01.181287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.182001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:01.182001 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.182001 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:01.182001 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:01.182001 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.182001 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:01.182001 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.182001 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:01.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:01.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:01.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:01.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:01.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:01.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:01.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:01.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:01.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:01.182792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.183192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:01.183192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:01.183192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:01.183192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:01.183192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:01.183192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:01.183192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:01.183192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:01.183192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.183192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:01.183192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:01.183192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.183192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:01.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:01.183727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:01.184254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:01.184795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:01.184795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:01.184795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:05.668974 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:05.669886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:05.669886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:05.671012 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:05.705247 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:05.705440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:05.705966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:05.705966 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:05.705966 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:05.705966 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:05.705966 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:05.705966 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:05.705966 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:05.705966 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:05.706624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:05.706624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:05.706624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:05.706624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:05.706624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:05.706624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:05.706624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:05.706624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:05.706624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:05.706624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:05.706624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:05.706624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:05.707185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:05.707185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:05.707185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:05.707185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:05.707185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:05.707185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:05.707185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:05.707185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:05.707185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:05.707185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:05.707185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:05.707185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:05.707185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:05.707748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:05.708440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:05.709255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:05.709255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:05.709255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:05.709255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:05.709255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:05.709255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:06.126452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:06.127058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:06.127269 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:06.128070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:06.128262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:06.128397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.128992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:06.128992 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.128992 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:06.128992 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:06.128992 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.128992 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:06.128992 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.128992 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:06.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:06.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:06.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:06.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:06.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:06.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:06.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:06.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:06.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:06.130406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.130962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:06.130962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:06.130962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:06.130962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:06.130962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:06.130962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:06.130962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:06.130962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:06.130962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.130962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:06.130962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:06.130962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.130962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:06.131554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.131554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:06.131980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:06.132684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.133526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:06.133526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.133526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:06.133667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:06.133982 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:06.134062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:06.134223 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:35:06.134466 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:06.134631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.135037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:06.135037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.135037 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:06.135037 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:06.135037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.135037 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:06.135037 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.135037 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:06.135786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:06.136452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:06.136452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:06.136452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.136452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:06.136452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:06.136452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.136452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:06.136452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.136452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:06.136865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.136865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:06.136865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:06.136865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:06.136865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:06.136865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:06.136865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:06.136865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:35:06.136865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:06.136865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:06.136865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.136865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:06.137436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:06.138259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:06.138857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:06.139679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:06.139679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:06.139679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:06.139679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:06.139679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:06.139679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.139679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:06.139679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:06.139679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.139679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:06.139679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.139679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:06.166767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:06.167129 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:06.167329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:06.167516 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:06.167817 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:06.168013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.168527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:06.168527 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.168527 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:06.168527 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:06.168527 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.168527 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:06.168527 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.168527 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:06.169655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:06.170346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:06.170346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:06.170346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.170346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:06.170346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:06.170346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.170346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:06.170346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.170346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:06.171009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.171009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:06.171009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:06.171009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:06.171009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:06.171009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:06.171009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:06.171009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:06.171009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:06.171009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.171009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:06.171522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:06.172117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.172117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:06.172117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.172117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:06.172628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:06.173183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:06.173750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.173750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:06.173750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.173750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:06.180657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:06.181022 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:06.181186 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:06.181443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:06.181699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:06.181890 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:06.182034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.182481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:06.182481 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.182481 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:06.182481 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:06.182481 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.182481 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:06.182481 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.182481 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:06.183294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.183294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:06.183294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:06.183294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:06.183294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:06.183294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:06.183294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:06.183294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.183294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:06.183294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:06.183294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:06.183294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.183823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:06.183823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:06.183823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:06.183823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:06.183823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:06.183823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:06.183823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:06.183823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:06.183823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.183823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:06.183823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:06.183823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.183823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:06.184398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:06.185050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:06.185895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:06.185895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:06.185895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:06.185895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:06.185895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:06.185895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:10.669680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:10.670524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:10.670524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:10.671680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:10.708468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:10.708785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:10.709811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:10.709811 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:10.709811 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:10.709811 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:10.709811 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:10.709811 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:10.709811 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:10.709811 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:10.710734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:10.710734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:10.710734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:10.710734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:10.710734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:10.710734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:10.710734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:10.710734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:10.710734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:10.710734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:10.710734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:10.710734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:10.711032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:10.711032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:10.711032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:10.711032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:10.711032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:10.711032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:10.711032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:10.711032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:10.711032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:10.711032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:10.711032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:10.711032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:10.711032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:10.711303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:10.711303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:10.711829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:10.712166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:10.712553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:10.712553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:10.712553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:11.127102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:11.127744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:11.127931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:11.134085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:11.137097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:11.137097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:11.137473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.138973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:11.138973 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.138973 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:11.138973 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:11.138973 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.138973 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:11.138973 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.138973 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:11.140685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.011 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.140685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:11.140685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:11.140685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:11.140685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:11.140685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:11.140685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:11.140685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.140685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:11.140685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:11.140685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:11.140685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.141726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:11.141726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:11.141726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:11.141726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:11.141726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:11.141726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:11.141726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:11.141726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:11.141726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.141726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:11.141726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:11.141726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.141726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:11.142342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.142342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:11.144308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:11.145552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.146579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:11.146579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.146579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:11.146579 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:11.146579 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:11.146579 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:35:11.146579 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:11.146579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.147175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:11.147175 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.147175 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:11.147175 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:11.147175 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.147175 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:11.147175 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.147175 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:11.147669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:11.148274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:11.148274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:11.148274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.148274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:11.148274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:11.148274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.148274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:11.148274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.148274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:11.151734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.151734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:11.151734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:11.151734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:11.151734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:11.151734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:11.151734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:11.151734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:35:11.151734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:11.151734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:11.151734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.151734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:11.152460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:11.153110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:11.153794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:11.154536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:11.154536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:11.154536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:11.154536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:11.154536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:11.154536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.154536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:11.154536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:11.154536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.154536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:11.154536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.154536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:11.168431 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:11.168822 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:11.169002 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:11.169201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:11.169853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:11.170421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:11.174489 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.174489 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:11.174489 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:11.174489 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.174489 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:11.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:11.175748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:11.176148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:11.176148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:11.176148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.176148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:11.176148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:11.176148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.176148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:11.176148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.176148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:11.177219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.177219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:11.177219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:11.177219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:11.177219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:11.177219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:11.177219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:11.177219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:11.177219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:11.177219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.177219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:11.177550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:11.177922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.177922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:11.177922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.177922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:11.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:11.179460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:11.179928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.179928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:11.179928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.179928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:11.181019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:11.181255 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:11.181385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:11.181534 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:11.181854 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:11.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:11.182215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.182683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:11.182683 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.182683 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:11.182683 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:11.182683 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.182683 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:11.182683 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.182683 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:11.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:11.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:11.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:11.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:11.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:11.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:11.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:11.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:11.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:11.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.183843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:11.183843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:11.183843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:11.183843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:11.183843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:11.183843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:11.183843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:11.183843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:11.183843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.183843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:11.183843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:11.183843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.183843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:11.184191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.184191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:11.184423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:11.184856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:11.185343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:11.185343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:11.185343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:15.670197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:15.671066 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:15.671066 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:15.671978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:15.702605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:15.702947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:15.703671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:15.703671 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:15.703671 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:15.703671 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:15.703671 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:15.703671 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:15.703671 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:15.703671 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:15.704377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:15.704377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:15.704377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:15.704377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:15.704377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:15.704377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:15.704377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:15.704377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:15.704377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:15.704377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:15.704377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:15.704377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:15.704895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:15.704895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:15.704895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:15.704895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:15.704895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:15.704895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:15.704895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:15.704895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:15.704895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:15.704895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:15.704895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:15.704895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:15.704895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:15.705420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:15.706045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:15.706831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:15.706831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:15.706831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:15.706831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:15.706831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:15.706831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:16.127803 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:16.128427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:16.128615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:16.129621 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:16.129894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:16.130137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.130972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:16.130972 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.130972 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:16.130972 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:16.130972 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.130972 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:16.130972 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.130972 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:16.132427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.132427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:16.132427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:16.132427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:16.132427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:16.132427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:16.132427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:16.132427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.132427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:16.132427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:16.132427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:16.132427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.133134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:16.133134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:16.133134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:16.133134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:16.133134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:16.133134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:16.133134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:16.133134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:16.133134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.133134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:16.133134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:16.133134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.133134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:16.133842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.133842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:16.134523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:16.136769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.137878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:16.137878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.137878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:16.144788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:16.145073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:16.145158 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:16.145256 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:35:16.145759 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:16.146138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.146909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:16.146909 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.146909 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:16.146909 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:16.146909 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.146909 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:16.146909 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.146909 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:16.147435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:16.147716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:16.147716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:16.147716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.147716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:16.147716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:16.147716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.147716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:16.147716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.147716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:16.148032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.148032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:16.148032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:16.148032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:16.148032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:16.148032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:16.148032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:16.148032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:35:16.148032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:16.148032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:16.148032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.148032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:16.148626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:16.149195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:16.149477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:16.149796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:16.149796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:16.149796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:16.149796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:16.149796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:16.149796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.149796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:16.149796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:16.149796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.149796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:16.149796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.149796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:16.169735 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:16.169735 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:16.170002 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:16.170002 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:16.170732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:16.171133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.172255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:16.172255 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.172255 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:16.172255 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:16.172255 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.172255 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:16.172255 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.172255 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:16.173284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:16.174256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:16.174927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:16.174927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:16.174927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:16.174927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:16.174927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.174927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:16.174927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:16.174927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:16.174927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.174927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:16.174927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:16.174927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:16.174927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:16.175535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:16.175535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:16.175535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:16.175535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:16.175535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.175535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:16.175535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:16.175535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.175535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:16.175535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.175535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:16.175535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:16.176106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.179039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:16.181440 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:16.182003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:16.182119 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:16.182318 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:16.182625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:16.182846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:16.182991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.183794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:16.183794 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.183794 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:16.183794 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:16.183794 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.183794 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:16.183794 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.183794 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:16.186559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.186559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:16.186559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:16.186559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:16.186559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:16.186559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:16.186559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:16.186559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.186559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:16.186559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:16.186559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:16.186559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.186904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:16.186904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:16.186904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:16.186904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:16.186904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:16.186904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:16.186904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:16.186904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:16.186904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.186904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:16.186904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:16.186904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.186904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:16.187227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.187227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:16.188602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:16.189004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:16.189476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:16.189476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:16.189476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:20.670982 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:20.671947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:20.671947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:20.672900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:20.709435 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:20.709899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:20.710802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:20.710802 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:20.710802 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:20.710802 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:20.710802 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:20.710802 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:20.710802 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:20.710802 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:20.712302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:20.712302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:20.712302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:20.712302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:20.712302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:20.712302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:20.712302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:20.712302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:20.712302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:20.712302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:20.712302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:20.712302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:20.713117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:20.713117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:20.713117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:20.713117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:20.713117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:20.713117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:20.713117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:20.713117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:20.713117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:20.713117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:20.713117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:20.713117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:20.713117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:20.713754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:20.718436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:20.718997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:20.718997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:20.718997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:20.718997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:20.718997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:20.718997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:21.128635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:21.128887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:21.129710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:21.130136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:21.130502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:21.131005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.131627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:21.131627 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.131627 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:21.131627 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:21.131627 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.131627 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:21.131627 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.131627 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:21.133178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.133178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:21.133178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:21.133178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:21.133178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:21.133178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:21.133178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:21.133178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.133178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:21.133178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:21.133178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:21.133178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.133759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:21.133759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:21.133759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:21.133759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:21.133759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:21.133759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:21.133759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:21.133759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:21.133759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.133759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:21.133759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:21.133759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.133759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:21.134249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.134249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:21.134931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:21.135649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.136122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:21.136122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.136122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:21.145315 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:21.145622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:21.145738 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:21.145868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:35:21.146225 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:21.146435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.147109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:21.147109 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.147109 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:21.147109 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:21.147109 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.147109 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:21.147109 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.147109 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:21.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:21.148986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:21.148986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:21.148986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.148986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:21.148986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:21.148986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.148986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:21.148986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.148986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:21.149892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.149892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:21.149892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:21.149892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:21.149892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:21.149892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:21.149892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:21.149892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:35:21.149892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:21.149892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:21.149892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.149892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:21.150322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:21.151600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.151600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:21.151600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.151600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:21.151942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:21.153314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:21.154012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.154012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:21.154012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:21.154012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.154012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:21.154012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.154012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:21.169763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:21.170391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:21.170522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:21.170702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:21.171142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:21.171416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.172016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:21.172016 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.172016 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:21.172016 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:21.172016 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.172016 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:21.172016 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.172016 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:21.173337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:21.173944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:21.173944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:21.173944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.173944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:21.173944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:21.173944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.173944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:21.173944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.173944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:21.174764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.174764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:21.174764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:21.174764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:21.174764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:21.174764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:21.174764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:21.174764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:21.174764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:21.174764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.174764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:21.176188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:21.177516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:21.178063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:21.178569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:21.178569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:21.178569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:21.178569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:21.178569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.178569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:21.178569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:21.178569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.178569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:21.178569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.178569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:21.181991 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:21.182334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:21.182460 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:21.182623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:21.183040 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:21.183398 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:21.183591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.184230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:21.184230 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.184230 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:21.184230 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:21.184230 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.184230 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:21.184230 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.184230 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:21.185110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.185110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:21.185110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:21.185110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:21.185110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:21.185110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:21.185110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:21.185110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.185110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:21.185110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:21.185110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:21.185110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.185531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:21.185531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:21.185531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:21.185531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:21.185531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:21.185531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:21.185531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:21.185531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:21.185531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.185531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:21.185531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:21.185531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.185531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:21.186000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.186000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:21.187957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:21.188380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:21.188851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:21.188851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:21.188851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:25.671815 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:25.672709 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:25.672709 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:25.673417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:25.713097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:25.713473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:25.714156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:25.714156 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:25.714156 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:25.714156 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:25.714156 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:25.714156 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:25.714156 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:25.714156 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:25.715325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:25.715325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:25.715325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:25.715325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:25.715325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:25.715325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:25.715325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:25.715325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:25.715325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:25.715325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:25.715325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:25.715325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:25.715960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:25.715960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:25.715960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:25.715960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:25.715960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:25.715960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:25.715960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:25.715960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:25.715960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:25.715960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:25.715960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:25.715960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:25.715960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:25.716625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:25.717388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:25.718420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:25.718420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:25.718420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:25.718420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:25.718420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:25.718420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:26.129142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:26.129480 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:26.129548 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:26.130615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:26.130966 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:26.131242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.132007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:26.132007 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.132007 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:26.132007 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:26.132007 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.132007 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:26.132007 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.132007 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:26.133286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.133286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:26.133286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:26.133286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:26.133286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:26.133286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:26.133286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:26.133286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.133286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:26.133286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:26.133286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:26.133286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.133911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:26.133911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:26.133911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:26.133911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:26.133911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:26.133911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:26.133911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:26.133911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:26.133911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.133911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:26.133911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:26.133911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.133911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:26.134655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.134655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:26.134915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:26.135589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.136127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:26.136127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.136127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:26.145774 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:26.146048 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:26.146159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:26.146321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:35:26.146717 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:26.146941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.147516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:26.147516 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.147516 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:26.147516 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:26.147516 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.147516 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:26.147516 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.147516 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:26.148250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:26.148746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:26.148746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:26.148746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.148746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:26.148746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:26.148746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.148746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:26.148746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.148746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:26.149019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.149019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:26.149019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:26.149019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:26.149019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:26.149019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:26.149019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:26.149019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:35:26.149019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:26.149019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:26.149019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.149019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:26.149442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:26.149819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.149819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:26.149819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.149819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:26.150566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:26.151079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:26.151723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.151723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:26.151723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:26.151723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.151723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:26.151723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.151723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:26.170385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:26.170752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:26.170863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:26.171075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:26.171541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:26.171933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.172848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:26.172848 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.172848 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:26.172848 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:26.172848 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.172848 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:26.172848 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.172848 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:26.173586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:26.174034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:26.174034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:26.174034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.174034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:26.174034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:26.174034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.174034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:26.174034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.174034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:26.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:26.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:26.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:26.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:26.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:26.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:26.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:26.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:26.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.174489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:26.174857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:26.175287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.175287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:26.175287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.175287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:26.175704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:26.176259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:26.176779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.176779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:26.176779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.176779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:26.182518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:26.182810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:26.182921 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:26.183067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:26.183528 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:26.183875 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:26.184101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.184889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:26.184889 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.184889 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:26.184889 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:26.184889 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.184889 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:26.184889 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.184889 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:26.186277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.186277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:26.186277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:26.186277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:26.186277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:26.186277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:26.186277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:26.186277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.186277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:26.186277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:26.186277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:26.186277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.186666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:26.186666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:26.186666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:26.186666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:26.186666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:26.186666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:26.186666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:26.186666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:26.186666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.186666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:26.186666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:26.186666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.186666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:26.186994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.186994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:26.187921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:26.188416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:26.188959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:26.188959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:26.188959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:30.672432 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:30.674369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:30.674369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:30.674638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:30.710806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:30.711182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:30.712347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:30.712347 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:30.712347 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:30.712347 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:30.712347 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:30.712347 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:30.712347 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:30.712347 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:30.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:30.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:30.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:30.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:30.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:30.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:30.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:30.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:30.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:30.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:30.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:30.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:30.715226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:30.715226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:30.715226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:30.715226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:30.715226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:30.715226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:30.715226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:30.715226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:30.715226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:30.715226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:30.715226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:30.715226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:30.715226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:30.715864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:30.716593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:30.718086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:30.718086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:30.718086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:30.718086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:30.718086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:30.718086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:31.129383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:31.129771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:31.129899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:31.131152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:31.131301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:31.131464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.132138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:31.132138 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.132138 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:31.132138 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:31.132138 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.132138 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:31.132138 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.132138 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:31.133615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.133615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:31.133615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:31.133615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:31.133615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:31.133615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:31.133615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:31.133615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.133615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:31.133615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:31.133615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:31.133615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.134269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:31.134269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:31.134269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:31.134269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:31.134269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:31.134269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:31.134269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:31.134269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:31.134269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.134269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:31.134269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:31.134269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.134269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:31.134794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:31.135286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.136050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:31.136050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:31.136050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.136050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:31.136050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.136050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:31.146320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:31.146612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:31.146725 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:31.146845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:35:31.147583 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:31.147904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.148826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:31.148826 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.148826 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:31.148826 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:31.148826 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.148826 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:31.148826 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.148826 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:31.150092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:31.150973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:31.150973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:31.150973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.150973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:31.150973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:31.150973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.150973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:31.150973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.150973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:31.151587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.151587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:31.151587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:31.151587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:31.151587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:31.151587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:31.151587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:31.151587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:35:31.151587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:31.151587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:31.151587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.151587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:31.151857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:31.152144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.152144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:31.152144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.152144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:31.153500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:31.153803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:31.154142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.154142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:31.154142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:31.154142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.154142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:31.154142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.154142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:31.170818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:31.171168 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:31.171390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:31.171654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:31.172234 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:31.172661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.173564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:31.173564 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.173564 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:31.173564 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:31.173564 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.173564 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:31.173564 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.173564 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:31.174432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:31.175118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:31.175118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:31.175118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.175118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:31.175118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:31.175118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.175118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:31.175118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.175118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:31.175565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.175565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:31.175565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:31.175565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:31.175565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:31.175565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:31.175565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:31.175565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:31.175565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:31.175565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.175565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:31.176023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:31.176639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:31.177302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:31.178251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:31.178251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:31.178251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:31.178251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:31.178251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.178251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:31.178251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:31.178251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.178251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:31.178251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.178251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:31.182845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:31.183097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:31.183221 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:31.183427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:31.183935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:31.184326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:31.184565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.185297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:31.185297 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.185297 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:31.185297 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:31.185297 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.185297 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:31.185297 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.185297 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:31.186220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.186220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:31.186220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:31.186220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:31.186220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:31.186220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:31.186220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:31.186220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.186220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:31.186220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:31.186220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:31.186220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.186572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:31.186572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:31.186572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:31.186572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:31.186572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:31.186572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:31.186572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:31.186572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:31.186572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.186572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:31.186572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:31.186572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.186572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:31.186979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.186979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:31.187272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:31.187883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:31.188746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:31.188746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:31.188746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:35.673548 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:35.676227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:35.676227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:35.676670 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:35.713097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:35.713503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:35.714333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:35.714333 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:35.714333 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:35.714333 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:35.714333 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:35.714333 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:35.714333 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:35.714333 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:35.717229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:35.717229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:35.717229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:35.717229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:35.717229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:35.717229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:35.717229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:35.717229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:35.717229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:35.717229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:35.717229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:35.717229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:35.717519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:35.717519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:35.717519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:35.717519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:35.717519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:35.717519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:35.717519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:35.717519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:35.717519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:35.717519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:35.717519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:35.717519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:35.717519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:35.717782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:35.718087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:35.718469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:35.718469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:35.718469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:35.718469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:35.718469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:35.718469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:36.129913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:36.130449 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:36.130641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:36.131713 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:36.132099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:36.136273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:36.136811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:36.137164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.137164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:36.137164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:36.137164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.137164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:36.137164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.137164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:36.139561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:36.139989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.140464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:36.140464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.140464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:36.146762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:36.147062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:36.147184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:36.148305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:35:36.148305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:36.148305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.148788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:36.148788 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.148788 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:36.148788 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:36.148788 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.148788 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:36.148788 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.148788 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:36.149495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:36.150038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:36.150038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:36.150038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.150038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:36.150038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:36.150038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.150038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:36.150038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.150038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:36.150540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.150540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:36.150540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:36.150540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:36.150540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:36.150540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:36.150540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:36.150540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:35:36.150540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:36.150540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:36.150540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.150540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:36.151089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:36.151675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:36.152381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:36.154802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:36.154802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:36.154802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:36.154802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:36.154802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:36.154802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.154802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:36.154802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:36.154802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.154802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:36.154802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.154802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:36.171256 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:36.171578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:36.171731 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:36.171857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:36.172330 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:36.172705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.173275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:36.173275 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.173275 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:36.173275 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:36.173275 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.173275 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:36.173275 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.173275 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:36.174319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:36.175177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:36.175766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:36.175766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:36.175766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:36.175766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:36.175766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.175766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:36.175766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:36.175766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:36.175766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.175766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:36.175766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:36.175766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:36.175766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:36.176307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:36.176307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:36.176307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:36.176307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:36.176307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.176307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:36.176307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:36.176307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.176307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:36.176307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.176307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:36.176307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:36.176861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.178557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:36.183399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:36.183721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:36.183824 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:36.183963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:36.184327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:36.184636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:36.184836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.185361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:36.185361 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.185361 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:36.185361 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:36.185361 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.185361 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:36.185361 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.185361 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:36.186087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.186087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:36.186087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:36.186087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:36.186087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:36.186087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:36.186087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:36.186087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.186087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:36.186087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:36.186087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:36.186087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.186474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:36.186474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:36.186474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:36.186474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:36.186474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:36.186474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:36.186474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:36.186474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:36.186474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.186474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:36.186474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:36.186474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.186474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:36.186785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.186785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:36.187109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:36.187639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:36.188279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:36.188279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:36.188279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:40.675481 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:40.676340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:40.676501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:40.677415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:40.683431 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:35:40.713293 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:40.713710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:40.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:40.714589 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:40.714589 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:40.714589 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:40.714589 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:40.714589 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:40.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:40.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:40.715564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:40.715564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:40.715564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:40.715564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:40.715564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:40.715564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:40.715564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:40.715564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:40.715564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:40.715564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:40.715564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:40.715564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:40.715931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:40.715931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:40.715931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:40.715931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:40.715931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:40.715931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:40.715931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:40.715931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:40.715931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:40.715931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:40.715931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:40.715931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:40.715931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:40.716291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:40.716291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:40.716667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:40.717448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:40.718287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:40.718287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:40.718287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:41.130661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:41.131226 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:41.131481 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:41.132622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:41.133035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:41.133313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.134174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:41.134174 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.134174 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:41.134174 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:41.134174 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.134174 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:41.134174 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.134174 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:41.136381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.136381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:41.136381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:41.136381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:41.136381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:41.136381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:41.136381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:41.136381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.136381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:41.136381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:41.136381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:41.136381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.137677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:41.137677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:41.137677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:41.137677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:41.137677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:41.137677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:41.137677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:41.137677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:41.137677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.137677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:41.137677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:41.137677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.137677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:41.138256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:41.138897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.141098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:41.141098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:41.141098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.141098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:41.141098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.141098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:41.147348 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:41.147728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:41.147728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:41.147891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:35:41.148339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:41.148623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.149350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:41.149350 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.149350 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:41.149350 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:41.149350 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.149350 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:41.149350 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.149350 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:41.150224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:41.150728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:41.150728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:41.150728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.150728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:41.150728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:41.150728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.150728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:41.150728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.150728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:41.151502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.151502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:41.151502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:41.151502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:41.151502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:41.151502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:41.151502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:41.151502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:35:41.151502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:41.151502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:41.151502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.151502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:41.151867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:41.152389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.152389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:41.152389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.152389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:41.153048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:41.153601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:41.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:41.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:41.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:41.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.154086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:41.171806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:41.172191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:41.172415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:41.172606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:41.173109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:41.173515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.174222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:41.174222 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.174222 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:41.174222 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:41.174222 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.174222 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:41.174222 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.174222 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:41.175382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:41.176047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:41.176047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:41.176047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.176047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:41.176047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:41.176047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.176047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:41.176047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.176047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:41.176728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.176728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:41.176728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:41.176728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:41.176728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:41.176728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:41.176728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:41.176728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:41.176728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:41.176728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.176728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:41.177237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:41.177842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.177842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:41.177842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.177842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:41.178416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:41.179147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:41.179923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.179923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:41.179923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.179923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:41.183986 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:41.184293 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:41.184401 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:41.184548 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:41.184974 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:41.185372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:41.185627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.186309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:41.186309 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.186309 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:41.186309 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:41.186309 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.186309 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:41.186309 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.186309 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:41.187083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.187083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:41.187083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:41.187083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:41.187083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:41.187083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:41.187083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:41.187083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.187083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:41.187083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:41.187083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:41.187083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.187453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:41.187453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:41.187453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:41.187453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:41.187453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:41.187453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:41.187453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:41.187453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:41.187453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.187453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:41.187453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:41.187453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.187453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:41.187824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.187824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:41.188033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:41.188498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:41.189012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:41.189012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:41.189012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:45.676182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:45.677071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:45.677071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:45.678316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:45.721235 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:45.721480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.722131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:45.722131 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:45.722131 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:45.722131 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:45.722131 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:45.722131 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:45.722131 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.722131 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:45.723200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.723200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:45.723200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:45.723200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:45.723200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:45.723200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:45.723200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:45.723200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:45.723200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:45.723200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:45.723200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:45.723200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:45.723770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:45.723770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:45.723770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:45.723770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:45.723770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:45.723770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:45.723770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:45.723770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:45.723770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:45.723770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:45.723770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:45.723770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:45.723770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:45.724369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.724369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:45.724653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:45.725187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:45.725948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:45.725948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.725948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:45.729125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:45.729443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:45.729595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:45.738110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 07 15:35:45.738278 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, security_groups=2, routers=1, router_ports=1 {{(pid=105503) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 07 15:35:45.738520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:35:45.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, resource={'id': '8b456c85-5e2d-47f8-9886-ebf65932cf32', 'name': 'private', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4a00c537-ed4f-405c-b241-82e7bc2cf3fa', '9fc89727-343d-403a-acd7-db6b4950e7b1'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:25:31Z', 'updated_at': '2026-05-07T15:25:34Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 42799}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:45.772634 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:45.772775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-b0f8152d-626b-440e-9383-919ec905ab38, external_ids_update=None, columns={'tag': 42799}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:45.772879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.773303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:45.773303 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:45.773303 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:45.773303 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:45.773303 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:45.773303 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:45.773303 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.773303 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:45.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:45.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:35:45.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:35:45.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:35:45.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:35:45.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:35:45.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:45.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:45.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:45.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:45.774402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:45.774697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:45.774697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.774697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:45.774697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:35:45.803417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, resource={'id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'name': 'public', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6aa26064-0a62-428b-a8ed-fef52fc73984', 'adf79775-4dfa-4110-bca0-59be9560299c'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-07T15:25:37Z', 'updated_at': '2026-05-07T15:25:45Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:45.803606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:45.803992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-2b8b3cbc-130c-4894-94e1-ab0729ec7077, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:45.804240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.804729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:45.804729 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:45.804729 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:45.804729 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:45.804729 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:45.804729 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:45.804729 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.804729 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:45.805765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.805765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:45.805765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:35:45.805765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:35:45.805765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:35:45.805765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:35:45.805765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:35:45.805765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:45.805765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:45.805765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:45.805765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:45.806350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:45.807012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:45.807012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.807012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:45.807012 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:35:45.833595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, resource={'id': '71aea053-d575-4d13-afc5-e6241ad6a15f', 'name': 'admin_net', 'tenant_id': '74d9c701d5cb4de6a15233057b01e413', 'project_id': '74d9c701d5cb4de6a15233057b01e413', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8e8a2cf7-97a7-4d62-ae4a-fa6c2139e811'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:30:51Z', 'updated_at': '2026-05-07T15:30:52Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 51150}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:45.833775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:45.834086 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d5042c49-4c90-465a-b356-c18cf966a58b, external_ids_update=None, columns={'tag': 51150}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:45.834190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.834673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:45.834673 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:45.834673 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:45.834673 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:45.834673 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:45.834673 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:45.834673 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.834673 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:45.835747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.835747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:45.835747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:35:45.835747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:35:45.835747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:35:45.835747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:35:45.835747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:35:45.835747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:45.835747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:45.835747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:45.835747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:45.836123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:45.836496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:45.836496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.836496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:45.836496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:35:45.862294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, resource={'id': '4fb571ad-ed2f-4edf-a199-cb52ff142698', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'tenant_id': '91e1e016e0f8434890282eb48ed58760', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2e6720c7-3159-41b9-80a2-fa6cf6a18808'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:31:26Z', 'updated_at': '2026-05-07T15:31:27Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 29129}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:45.862483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:45.862705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13b8a284-1038-44ba-86e4-667f771dfae2, external_ids_update=None, columns={'tag': 29129}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:45.862785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.863333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:45.863333 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:45.863333 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:45.863333 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:45.863333 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:45.863333 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:45.863333 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.863333 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:45.864378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.864378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:45.864378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:35:45.864378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:35:45.864378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:35:45.864378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:35:45.864378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:35:45.864378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:45.864378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:45.864378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:45.864378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:45.864855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:45.865599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:45.865599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:45.865599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:45.865599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 5b41b656-9cce-46a8-ba7f-088502107b4d (type: security_groups) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:35:46.131371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:46.184532 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:46.184532 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:46.184532 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:47.221568 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.db.ovn_revision_numbers_db [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Successfully bumped revision number for resource 5b41b656-9cce-46a8-ba7f-088502107b4d (type: security_groups) to 10
May 07 15:35:47.222330 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource b865f7dd-b4ca-4283-b75a-4d46c27e390a (type: security_groups) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:35:48.444404 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.db.ovn_revision_numbers_db [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Successfully bumped revision number for resource b865f7dd-b4ca-4283-b75a-4d46c27e390a (type: security_groups) to 15
May 07 15:35:48.444404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:35:48.533388 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, resource={'id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'enable_snat': True}, 'gw_port_id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-jseci', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'ip_address': '172.24.5.213', 'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984'}, {'ip_address': '2001:db8::15a', 'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 3, 'tenant_id': '91e1e016e0f8434890282eb48ed58760'}, resource_type=routers, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:48.533735 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:48.533735 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:48.533951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:48.534042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.534829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:48.534829 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.534829 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:48.534829 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:48.534829 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.534829 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:48.534829 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.534829 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:48.534829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Unable to update router 0938bfb8-1052-49e2-bd69-e5fda45e8a2f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.536492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.536492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:48.536492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:35:48.536492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:35:48.536492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:35:48.536492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:35:48.536492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 07 15:35:48.536492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 07 15:35:48.536492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 07 15:35:48.536492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 07 15:35:48.536492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:48.537909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:48.538673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:48.538673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:48.538673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.538673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:48.538673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:48.538673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.538673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:48.538673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.538673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:48.538673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:35:48.662667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, resource={'id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'name': '', 'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'project_id': '', 'mac_address': 'fa:16:3e:f6:db:65', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000172501', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:48.662851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '6aa26064-0a62-428b-a8ed-fef52fc73984 adf79775-4dfa-4110-bca0-59be9560299c', 'neutron:network_name': 'neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.213/24', '2001:db8::15a/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:48.662851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=b30ffafe-8df3-4064-bcbf-e3d1daad619c, lrouter_port=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:48.663049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.663588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:48.663588 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.663588 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:48.663588 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:48.663588 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.663588 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:48.663588 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.663588 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:48.664636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.664636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:48.664636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:35:48.664636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:35:48.664636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:35:48.664636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:35:48.664636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 07 15:35:48.664636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:48.664636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.664636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:48.664636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:48.665235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.665857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:48.665857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.665857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:48.665857 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_for_inconsistencies finished in 2.935 seconds
May 07 15:35:48.665857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:48.665857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:48.665857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:48.666189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:48.666189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.666189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:48.666189 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.666189 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:48.666189 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:48.666189 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.666189 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:48.666189 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.666189 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:48.666725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.666725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:48.666725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:48.666725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:48.666725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:48.666725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:48.666725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:48.666725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.666725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:48.666725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:48.666725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:48.666725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.667275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:48.667275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:48.667275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:48.667275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:48.667275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:48.667275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:48.667275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:48.667275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:48.667275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.667275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:48.667275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:48.667275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.667275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:48.667794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:48.668534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.669372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:48.669372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:48.669372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.669372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:48.669372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.669372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:48.669372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:48.669372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:48.669372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:35:48.669372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:48.669848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.670659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:48.671416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.671416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:48.671416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:48.671416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:48.671416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:48.671416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:48.671416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:48.671416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:35:48.671416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:48.671416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:48.671416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.671416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:48.671959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:48.672670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:48.673125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:48.673522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:48.673863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:48.673863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:48.673863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.673863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:48.673863 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.673863 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:48.673863 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:48.673863 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.673863 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:48.673863 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.673863 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:48.673863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:48.674116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.674449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:48.674449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:48.674449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.674449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:48.674449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.674449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:48.674449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.674449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:48.674449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:48.674449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:48.674449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:48.674449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:48.674449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:48.674732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:48.674732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:48.674732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.674732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:48.674732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:48.674732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:48.674732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.674732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:48.674732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:48.674732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:48.674732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:48.674732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:48.674732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:48.675019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:48.675019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:48.675019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.675019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:48.675019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:48.675019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.675019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:48.675019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.675019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:48.675019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.675019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:48.675019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.675281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.675647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:48.675992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:48.675992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:48.675992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:48.675992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:48.675992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:48.675992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:48.676132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:48.676525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:48.676794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:48.677072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:48.677478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:48.677478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:48.677478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:48.677478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:48.677478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:48.677478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:48.677478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:48.677478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:48.677478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:48.677478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:50.676858 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:50.677854 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:50.678076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:50.679492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:50.713607 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:50.714094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:50.715280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:50.715280 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:50.715280 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:50.715280 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:50.715280 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:50.715280 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:50.715280 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:50.715280 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:50.716029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:50.716029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:50.716029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:50.716029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:50.716029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:50.716029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:50.716029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:50.716029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:50.716029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:50.716029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:50.716029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:50.716029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:50.716591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:50.716591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:50.716591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:50.716591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:50.716591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:50.716591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:50.716591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:50.716591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:50.716591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:50.716591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:50.716591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:50.716591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:50.716591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:50.717166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:50.717842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:50.718839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:50.718839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:50.718839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:50.718839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:50.718839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:50.718839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:53.717473 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:53.717473 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:53.717473 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:53.717473 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:53.717473 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:53.717473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:53.720687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:53.721707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:53.721707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:53.721707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.721707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:53.721707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:53.721707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:53.721707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.721707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:53.721707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:53.721707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:53.721707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:53.721707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:53.721707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:53.722362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:53.722362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:53.722362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.722362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:53.722362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:53.722362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.722362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:53.722362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.722362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:53.722362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:53.723079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:53.724091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.725256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:53.725256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.725256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:53.725256 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:53.725256 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:53.725256 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:53.725256 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:35:53.725256 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:53.725751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.726672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:53.727451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.727451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:53.727451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:53.727451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:53.727451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:53.727451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:53.727451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:53.727451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:35:53.727451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:53.727451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:53.727451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.727451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:53.727727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:53.727980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:53.728719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:53.729078 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:53.729361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:53.729361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:53.729361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.729361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:53.729361 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.729361 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:53.729361 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:53.729361 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.729361 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:53.729361 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.729361 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:53.729361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:53.729578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:53.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:53.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:53.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:53.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:53.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:53.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:53.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:53.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:53.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:53.730114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:53.730114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:53.730114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.730114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:53.730114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:53.730114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:53.730114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.730114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:53.730114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:53.730114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:53.730114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:53.730114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:53.730114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:53.730364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:53.730364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:53.730364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.730364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:53.730364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:53.730364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.730364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:53.730364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.730364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:53.730364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.730364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:53.730364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.730600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.730949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:53.731337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:53.731337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:53.731337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:53.731337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:53.731337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:53.731337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:53.731494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:53.731795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:53.732070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:53.732385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:53.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:53.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:53.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:53.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:53.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:53.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:53.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:53.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:53.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:53.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:55.677825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:55.678580 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:55.678580 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:55.678925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:55.712502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:55.712922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:55.713083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:55.713083 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:55.713083 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:55.713083 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:55.713083 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:55.713083 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:55.713083 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:55.713083 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:55.713701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:55.713701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:55.713701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:55.713701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:55.713701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:55.713701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:55.713701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:55.713701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:55.713701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:55.713701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:55.713701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:55.713701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:55.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:55.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:55.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:55.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:55.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:55.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:55.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:55.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:55.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:55.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:55.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:55.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:55.714589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:55.715328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:55.716347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:55.717020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:55.717020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:55.717020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:55.717020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:55.717020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:55.717020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:58.666104 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:58.666775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:58.666972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:58.667895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:58.668173 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:58.668457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.669356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:58.669356 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.669356 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:58.669356 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:58.669356 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.669356 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:58.669356 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.669356 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:58.670275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.670275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:58.670275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:58.670275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:58.670275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:58.670275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:58.670275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:58.670275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.670275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:58.670275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:58.670275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:58.670275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.670801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:58.670801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:58.670801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:58.670801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:58.670801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:58.670801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:58.670801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:58.670801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:58.670801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.670801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:58.670801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:58.670801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.670801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:58.671356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:58.671981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.672795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:58.672795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:58.672795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.672795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:58.672795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.672795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:58.674967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:58.675249 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:58.675366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:58.675479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:35:58.675720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:58.675888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.676338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:58.676338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.676338 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:58.676338 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:58.676338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.676338 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:58.676338 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.676338 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:58.677042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:58.677524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:58.677524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:58.677524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.677524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:58.677524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:58.677524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.677524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:58.677524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.677524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:58.677884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.677884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:58.677884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:58.677884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:58.677884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:58.677884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:58.677884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:58.677884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:35:58.677884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:58.677884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:58.677884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.677884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:58.678252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:58.678634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.678634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:58.678634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.678634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:58.678844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:58.679293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:58.679776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.679776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:58.679776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:58.679776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.679776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:58.679776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.679776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:58.680053 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:58.680295 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:58.680398 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:58.680518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:58.680770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:58.680975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.681392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:58.681392 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.681392 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:58.681392 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:58.681392 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.681392 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:58.681392 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.681392 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:35:58.682048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:35:58.682469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:58.682469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:35:58.682469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.682469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:35:58.682469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:35:58.682469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.682469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:35:58.682469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.682469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:35:58.684502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.684502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:58.684502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:58.684502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:58.684502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:58.684502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:58.684502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:58.684502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:58.684502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:58.684502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.684502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:58.685694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:58.686228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:58.686730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:58.687478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:58.687478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:58.687478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:58.687478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:58.687478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.687478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:58.687478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:58.687478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.687478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:58.687478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.687478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:35:58.687478 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:35:58.687478 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:35:58.687979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:35:58.687979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:35:58.687979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:58.687979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:35:58.687979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.687979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:35:58.687979 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.687979 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:35:58.687979 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:58.688350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:35:58.688958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:35:58.689536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:35:58.690226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:35:58.690997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:35:58.690997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:35:58.690997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:35:58.690997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:35:58.690997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:35:58.690997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:00.678629 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:00.679575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:00.679575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:00.680563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:00.712948 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:00.713353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:00.714307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:00.714307 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:00.714307 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:00.714307 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:00.714307 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:00.714307 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:00.714307 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:00.714307 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:00.715395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:00.715395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:00.715395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:00.715395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:00.715395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:00.715395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:00.715395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:00.715395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:00.715395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:00.715395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:00.715395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:00.715395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:00.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:00.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:00.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:00.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:00.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:00.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:00.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:00.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:00.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:00.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:00.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:00.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:00.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:00.716479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:00.716479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:00.716597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:00.717278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:00.718044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:00.718044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:00.718044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:03.666743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:03.667362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:03.667570 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:03.668717 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:03.669087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:03.669395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.670322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:03.670322 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.670322 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:03.670322 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:03.670322 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.670322 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:03.670322 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.670322 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:03.671864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.671864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:03.671864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:03.671864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:03.671864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:03.671864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:03.671864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:03.671864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.671864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:03.671864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:03.671864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:03.671864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.672483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:03.672483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:03.672483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:03.672483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:03.672483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:03.672483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:03.672483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:03.672483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:03.672483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.672483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:03.672483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:03.672483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.672483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:03.673057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.673057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:03.673650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:03.674076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.674625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:03.674625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.674625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:03.675398 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:03.675797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:03.675971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:03.676150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:36:03.676679 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:03.676964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.677738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:03.677738 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.677738 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:03.677738 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:03.677738 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.677738 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:03.677738 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.677738 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:03.678937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:03.679398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:03.679398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:03.679398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.679398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:03.679398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:03.679398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.679398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:03.679398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.679398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:03.679907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.679907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:03.679907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:03.679907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:03.679907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:03.679907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:03.679907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:03.679907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:36:03.679907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:03.679907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:03.679907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.679907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:03.680282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:03.680652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.680652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:03.680652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.680652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:03.681452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:03.681868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:03.682321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.682321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:03.682321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:03.682321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.682321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:03.682321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.682321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:03.682321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:03.682321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:03.682321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:03.682791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:03.682952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:03.683322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.683994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:03.683994 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.683994 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:03.683994 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:03.683994 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.683994 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:03.683994 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.683994 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:03.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:03.685137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:03.685137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:03.685137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.685137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:03.685137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:03.685137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.685137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:03.685137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.685137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:03.685704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.685704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:03.685704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:03.685704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:03.685704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:03.685704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:03.685704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:03.685704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:03.685704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:03.685704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.685704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:03.686016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:03.686480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.686480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:03.686480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.686480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:03.686796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:03.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:03.687831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.687831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:03.687831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.687831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:03.687831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:03.687831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:03.687831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:03.687831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:03.688068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:03.688405 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:03.688627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.689313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:03.689313 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.689313 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:03.689313 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:03.689313 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.689313 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:03.689313 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.689313 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:03.689883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.689883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:03.689883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:03.689883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:03.689883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:03.689883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:03.689883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:03.689883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.689883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:03.689883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:03.689883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:03.689883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.690229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:03.690229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:03.690229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:03.690229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:03.690229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:03.690229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:03.690229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:03.690229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:03.690229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.690229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:03.690229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:03.690229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.690229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:03.690584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:03.690977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:03.691460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:03.691460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:03.691460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:03.691460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:03.691460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:03.691460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:05.679158 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:05.679968 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:05.679968 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:05.680691 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:05.713940 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:05.714198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:05.714897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:05.714897 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:05.714897 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:05.714897 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:05.714897 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:05.714897 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:05.714897 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:05.714897 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:05.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:05.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:05.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:05.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:05.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:05.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:05.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:05.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:05.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:05.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:05.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:05.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:05.716415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:05.716415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:05.716415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:05.716415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:05.716415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:05.716415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:05.716415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:05.716415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:05.716415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:05.716415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:05.716415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:05.716415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:05.716415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:05.717115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:05.717962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:05.718974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:05.718974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:05.718974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:05.718974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:05.718974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:05.718974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:08.667519 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:08.668342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:08.668342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:08.670364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:08.670364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:08.670364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.670873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:08.670873 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.670873 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:08.670873 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:08.670873 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.670873 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:08.670873 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.670873 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:08.672387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.672387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:08.672387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:08.672387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:08.672387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:08.672387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:08.672387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:08.672387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.672387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:08.672387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:08.672387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:08.672387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.672977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:08.672977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:08.672977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:08.672977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:08.672977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:08.672977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:08.672977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:08.672977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:08.672977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.672977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:08.672977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:08.672977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.672977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:08.673545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.673545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:08.674041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:08.674785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.675586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:08.675586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.675586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:08.675791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:08.676124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:08.676334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:08.676515 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:36:08.676873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:08.677058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.677666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:08.677666 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.677666 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:08.677666 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:08.677666 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.677666 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:08.677666 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.677666 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:08.678434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:08.679146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:08.679146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:08.679146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.679146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:08.679146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:08.679146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.679146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:08.679146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.679146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:08.679606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.679606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:08.679606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:08.679606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:08.679606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:08.679606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:08.679606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:08.679606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:36:08.679606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:08.679606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:08.679606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.679606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:08.680149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:08.680771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:08.681387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:08.682315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:08.682315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:08.682315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:08.682315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:08.682315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:08.682315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.682315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:08.682315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:08.682315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.682315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:08.682315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.682315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:08.682694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:08.682694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:08.682775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:08.682850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:08.683177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:08.683415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.683995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:08.683995 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.683995 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:08.683995 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:08.683995 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.683995 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:08.683995 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.683995 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:08.684736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:08.685129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:08.685129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:08.685129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.685129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:08.685129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:08.685129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.685129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:08.685129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.685129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:08.685551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.685551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:08.685551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:08.685551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:08.685551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:08.685551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:08.685551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:08.685551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:08.685551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:08.685551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.685551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:08.685903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:08.686427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.686427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:08.686427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.686427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:08.686592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:08.687046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:08.687546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.687546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:08.687546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.687546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:08.687546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:08.687705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:08.687705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:08.687847 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:08.688272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:08.688598 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:08.688809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.689492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:08.689492 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.689492 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:08.689492 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:08.689492 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.689492 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:08.689492 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.689492 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:08.690318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.690318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:08.690318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:08.690318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:08.690318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:08.690318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:08.690318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:08.690318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.690318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:08.690318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:08.690318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:08.690318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.690706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:08.690706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:08.690706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:08.690706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:08.690706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:08.690706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:08.690706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:08.690706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:08.690706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.690706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:08.690706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:08.690706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.690706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:08.691050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.691050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:08.691263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:08.691705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:08.692185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:08.692185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:08.692185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:10.679939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:10.680850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:10.680850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:10.682063 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:10.955608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:10.955608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:10.955608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:10.955608 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:10.955608 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:10.955608 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:10.955608 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:10.955608 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:10.955608 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:10.955608 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:10.955608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.208 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:10.955608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:10.955608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:10.956609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:10.957459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:10.957459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:10.957459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:10.957459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:10.957459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:10.957459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:10.957459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:10.957459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:10.957459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:10.957459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:10.957459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:10.957459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:10.958071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:10.958976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:13.668076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:13.669876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:13.669876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:13.670841 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:13.671239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:13.671494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.672516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:13.672516 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.672516 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:13.672516 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:13.672516 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.672516 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:13.672516 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.672516 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:13.674174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.674174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:13.674174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:13.674174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:13.674174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:13.674174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:13.674174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:13.674174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.674174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:13.674174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:13.674174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:13.674174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.674751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:13.674751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:13.674751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:13.674751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:13.674751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:13.674751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:13.674751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:13.674751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:13.674751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.674751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:13.674751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:13.674751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.674751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:13.675320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.675320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:13.675455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:13.676111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.676915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:13.676915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.676915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:13.676915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:13.676915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:13.676915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:13.677230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:36:13.677511 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:13.677800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.678516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:13.678516 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.678516 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:13.678516 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:13.678516 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.678516 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:13.678516 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.678516 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:13.679414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:13.679855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:13.679855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:13.679855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.679855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:13.679855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:13.679855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.679855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:13.679855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.679855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:13.680314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.680314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:13.680314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:13.680314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:13.680314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:13.680314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:13.680314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:13.680314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:36:13.680314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:13.680314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:13.680314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.680314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:13.680658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:13.681033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.681033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:13.681033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.681033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:13.681342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:13.681762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:13.682241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.682241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:13.682241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:13.682241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.682241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:13.682241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.682241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:13.682725 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:13.683041 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:13.683156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:13.683308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:13.683872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:13.684327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.685068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:13.685068 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.685068 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:13.685068 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:13.685068 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.685068 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:13.685068 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.685068 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:13.685864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:13.686269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:13.686269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:13.686269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.686269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:13.686269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:13.686269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.686269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:13.686269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.686269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:13.686746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.686746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:13.686746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:13.686746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:13.686746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:13.686746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:13.686746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:13.686746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:13.686746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:13.686746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.686746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:13.687056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:13.687441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.687441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:13.687441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.687441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:13.687808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:13.688291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:13.688742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.688742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:13.688742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.688742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:13.688742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:13.688742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:13.688742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:13.688929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:13.689289 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:13.689627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:13.689851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.690630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:13.690630 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.690630 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:13.690630 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:13.690630 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.690630 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:13.690630 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.690630 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:13.691476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.691476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:13.691476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:13.691476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:13.691476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:13.691476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:13.691476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:13.691476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.691476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:13.691476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:13.691476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:13.691476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.691815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:13.691815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:13.691815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:13.691815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:13.691815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:13.691815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:13.691815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:13.691815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:13.691815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.691815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:13.691815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:13.691815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.691815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:13.692273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.692273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:13.692522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:13.692944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:13.693478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:13.693478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:13.693478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:15.681382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:15.682001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:15.682292 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:15.683446 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:15.716356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:15.716356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:15.716642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:15.716642 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:15.716642 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:15.716642 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:15.716642 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:15.716642 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:15.716642 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:15.716642 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:15.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:15.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:15.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:15.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:15.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:15.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:15.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:15.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:15.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:15.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:15.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:15.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:15.718466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:15.718466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:15.718466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:15.718466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:15.718466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:15.718466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:15.718466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:15.718466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:15.718466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:15.718466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:15.718466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:15.718466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:15.718466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:15.718992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:15.719988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:15.721007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:15.721007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:15.721007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:15.721007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:15.721007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:15.721007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:18.669108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:18.669930 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:18.669994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:18.671270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:18.671634 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:18.671906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.672878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:18.672878 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.672878 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:18.672878 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:18.672878 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.672878 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:18.672878 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.672878 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:18.674492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.674492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:18.674492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:18.674492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:18.674492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:18.674492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:18.674492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:18.674492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.674492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:18.674492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:18.674492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:18.674492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.674984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:18.674984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:18.674984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:18.674984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:18.674984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:18.674984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:18.674984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:18.674984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:18.674984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.674984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:18.674984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:18.674984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.674984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:18.675530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.675530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:18.675665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:18.676414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.677231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:18.677231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.677231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:18.677231 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:18.677231 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:18.677231 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:18.677535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:36:18.677822 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:18.678109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.678869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:18.678869 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.678869 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:18.678869 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:18.678869 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.678869 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:18.678869 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.678869 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:18.679656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:18.680384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:18.680384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:18.680384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.680384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:18.680384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:18.680384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.680384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:18.680384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.680384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:18.680824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.680824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:18.680824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:18.680824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:18.680824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:18.680824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:18.680824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:18.680824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:36:18.680824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:18.680824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:18.680824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.680824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:18.681395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:18.682013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:18.682459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:18.682961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:18.682961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:18.682961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:18.682961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:18.682961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:18.682961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.682961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:18.682961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:18.682961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.682961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:18.682961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.682961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:18.683309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:18.683387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:18.683492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:18.683616 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:18.684071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:18.684470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.685220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:18.685220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.685220 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:18.685220 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:18.685220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.685220 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:18.685220 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.685220 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:18.685978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:18.686424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:18.686424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:18.686424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.686424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:18.686424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:18.686424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.686424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:18.686424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.686424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:18.686894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.686894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:18.686894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:18.686894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:18.686894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:18.686894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:18.686894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:18.686894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:18.686894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:18.686894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.686894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:18.687197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:18.687582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.687582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:18.687582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.687582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:18.687970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:18.688447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:18.689036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.689036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:18.689036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.689036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:18.689036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:18.689036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:18.689036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:18.689250 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:18.689599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:18.689928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:18.690140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.690847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:18.690847 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.690847 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:18.690847 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:18.690847 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.690847 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:18.690847 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.690847 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:18.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:18.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:18.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:18.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:18.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:18.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:18.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:18.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:18.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:18.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.691993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:18.691993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:18.691993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:18.691993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:18.691993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:18.691993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:18.691993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:18.691993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:18.691993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.691993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:18.691993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:18.691993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.691993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:18.692379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.692379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:18.692667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:18.693111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:18.693674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:18.693674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:18.693674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:20.682123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:20.683000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:20.683000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:20.683610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:20.831100 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:20.831100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:20.831100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:20.831100 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:20.831100 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:20.831100 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:20.831100 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:20.831100 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:20.831100 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:20.831100 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:20.831100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.132 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:20.831100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:20.831100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:20.831946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:20.832581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:20.832581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:20.832581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:20.832581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:20.832581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:20.832581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:20.832581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:20.832581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:20.832581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:20.832581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:20.832581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:20.832581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:20.833095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:20.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:23.669796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:23.670535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:23.670605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:23.671459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:23.671676 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:23.671825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.672779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:23.672779 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.672779 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:23.672779 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:23.672779 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.672779 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:23.672779 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.672779 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:23.674308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.674308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:23.674308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:23.674308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:23.674308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:23.674308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:23.674308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:23.674308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.674308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:23.674308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:23.674308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:23.674308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.674929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:23.674929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:23.674929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:23.674929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:23.674929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:23.674929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:23.674929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:23.674929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:23.674929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.674929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:23.674929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:23.674929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.674929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:23.675562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.675562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:23.675993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:23.676833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.677797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:23.677797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.677797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:23.677797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:23.677797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:23.678087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:23.678087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:36:23.678530 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:23.678807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.679485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:23.679485 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.679485 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:23.679485 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:23.679485 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.679485 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:23.679485 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.679485 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:23.680361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:23.681100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:23.681100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:23.681100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.681100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:23.681100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:23.681100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.681100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:23.681100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.681100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:23.681548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.681548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:23.681548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:23.681548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:23.681548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:23.681548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:23.681548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:23.681548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:36:23.681548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:23.681548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:23.681548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.681548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:23.682125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:23.682921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:23.683657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:23.684613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:23.685118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:23.685118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:23.685118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:23.685118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.685118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:23.685118 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.685118 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:23.685118 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:23.685118 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.685118 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:23.685118 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.685118 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:23.685866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:23.686298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:23.686298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:23.686298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.686298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:23.686298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:23.686298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.686298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:23.686298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.686298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:23.686765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.686765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:23.686765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:23.686765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:23.686765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:23.686765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:23.686765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:23.686765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:23.686765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:23.686765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.686765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:23.687127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:23.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:23.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:23.687867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:23.688402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:23.688932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.688932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:23.688932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.688932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:23.689054 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:23.689348 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:23.689457 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:23.689640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:23.690069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:23.690432 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:23.690655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.691412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:23.691412 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.691412 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:23.691412 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:23.691412 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.691412 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:23.691412 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.691412 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:23.692460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.692460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:23.692460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:23.692460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:23.692460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:23.692460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:23.692460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:23.692460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.692460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:23.692460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:23.692460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:23.692460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.692903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:23.692903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:23.692903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:23.692903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:23.692903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:23.692903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:23.692903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:23.692903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:23.692903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.692903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:23.692903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:23.692903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.692903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:23.693281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.693281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:23.693543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:23.694020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:23.694573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:23.694573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:23.694573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:25.682880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:25.683691 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:25.683691 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:25.684750 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:25.720273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:25.729037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:25.729893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:25.730535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:25.730535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:25.730535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:25.730535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:25.730535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:25.730535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:25.730535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:25.730535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:25.730535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:25.730535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:25.730535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:25.730535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:25.730535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:25.731095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:25.731984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:25.731984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:25.731984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:25.731984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:25.731984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:25.731984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:25.731984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:25.731984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:25.731984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:25.731984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:25.731984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:25.731984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:28.670426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:28.671057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:28.671057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:28.672265 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:28.672639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:28.672930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.674553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:28.674553 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.674553 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:28.674553 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:28.674553 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.674553 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:28.674553 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.674553 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:28.676427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.676427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:28.676427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:28.676427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:28.676427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:28.676427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:28.676427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:28.676427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.676427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:28.676427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:28.676427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:28.676427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.676827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:28.676827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:28.676827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:28.676827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:28.676827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:28.676827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:28.676827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:28.676827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:28.676827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.676827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:28.676827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:28.676827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.676827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:28.677242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.677242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:28.679240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:28.679843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.680497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:28.680497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.680497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:28.680497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:28.680497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:28.680497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:28.680497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:36:28.680497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:28.680729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.681415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:28.681415 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.681415 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:28.681415 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:28.681415 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.681415 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:28.681415 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.681415 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:28.682131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:28.682694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:28.682694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:28.682694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.682694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:28.682694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:28.682694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.682694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:28.682694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.682694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:28.683067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.683067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:28.683067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:28.683067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:28.683067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:28.683067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:28.683067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:28.683067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:36:28.683067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:28.683067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:28.683067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.683067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:28.683546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:28.684062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:28.684649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:28.685095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:28.685474 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:28.685474 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:28.685474 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:28.685474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.685474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:28.685474 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.685474 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:28.685474 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:28.685474 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.685474 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:28.685474 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.685474 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:28.686062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:28.686748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:28.686748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:28.686748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.686748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:28.686748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:28.686748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.686748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:28.686748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.686748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:28.687105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.687105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:28.687105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:28.687105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:28.687105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:28.687105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:28.687105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:28.687105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:28.687105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:28.687105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.687105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:28.687543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:28.688048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:28.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:28.689346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:28.689346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:28.689346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:28.689346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:28.689346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.689346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:28.689346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:28.689346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.689346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:28.689346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.689346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:28.689773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:28.689773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:28.689773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:28.689773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:28.690030 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:28.690235 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:28.690380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.690822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:28.690822 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.690822 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:28.690822 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:28.690822 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.690822 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:28.690822 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.690822 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:28.691425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.691425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:28.691425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:28.691425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:28.691425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:28.691425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:28.691425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:28.691425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.691425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:28.691425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:28.691425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:28.691425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.691902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:28.691902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:28.691902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:28.691902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:28.691902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:28.691902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:28.691902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:28.691902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:28.691902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.691902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:28.691902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:28.691902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.691902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:28.692427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:28.693018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:28.693722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:28.693722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:28.693722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:28.693722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:28.693722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:28.693722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:30.683426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:30.684415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:30.684415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:30.685419 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:30.721954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:30.722467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:30.723383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:30.723383 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:30.723383 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:30.723383 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:30.723383 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:30.723383 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:30.723383 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:30.723383 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:30.724350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:30.724350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:30.724350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:30.724350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:30.724350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:30.724350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:30.724350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:30.724350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:30.724350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:30.724350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:30.724350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:30.724350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:30.724977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:30.724977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:30.724977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:30.724977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:30.724977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:30.724977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:30.724977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:30.724977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:30.724977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:30.724977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:30.724977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:30.724977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:30.724977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:30.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:30.726422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:30.727339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:30.727339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:30.727339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:30.727339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:30.727339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:30.727339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:33.671091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:33.671725 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:33.671917 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:33.672815 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:33.673065 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:33.673262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.674124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:33.674124 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.674124 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:33.674124 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:33.674124 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.674124 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:33.674124 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.674124 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:33.675680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.675680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:33.675680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:33.675680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:33.675680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:33.675680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:33.675680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:33.675680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.675680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:33.675680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:33.675680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:33.675680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.675943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:33.675943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:33.675943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:33.675943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:33.675943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:33.675943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:33.675943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:33.675943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:33.675943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.675943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:33.675943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:33.675943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.675943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:33.676190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.676190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:33.677426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:33.677737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.678086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:33.678086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.678086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:33.679811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:33.680106 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:33.680248 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:33.680372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:36:33.680846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:33.681151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.681890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:33.681890 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.681890 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:33.681890 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:33.681890 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.681890 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:33.681890 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.681890 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:33.682637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:33.682943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:33.682943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:33.682943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.682943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:33.682943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:33.682943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.682943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:33.682943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.682943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:33.683462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.683462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:33.683462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:33.683462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:33.683462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:33.683462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:33.683462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:33.683462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:36:33.683462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:33.683462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:33.683462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.683462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:33.683697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:33.683954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.683954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:33.683954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.683954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:33.684585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:33.684893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:33.685230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.685230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:33.685230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:33.685230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.685230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:33.685230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.685230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:33.685230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:33.685230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:33.685412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:33.685412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:33.685738 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:33.685960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.686577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:33.686577 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.686577 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:33.686577 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:33.686577 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.686577 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:33.686577 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.686577 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:33.687369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:33.687669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:33.687669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:33.687669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.687669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:33.687669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:33.687669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.687669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:33.687669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.687669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:33.688193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.688193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:33.688193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:33.688193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:33.688193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:33.688193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:33.688193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:33.688193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:33.688193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:33.688193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.688193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:33.688623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:33.688892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.688892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:33.688892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.688892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:33.689218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:33.689556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:33.689906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.689906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:33.689906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.689906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:33.689906 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:33.690009 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:33.690113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:33.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:33.690601 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:33.690848 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:33.690994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.691600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:33.691600 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.691600 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:33.691600 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:33.691600 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.691600 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:33.691600 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.691600 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:33.692423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.692423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:33.692423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:33.692423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:33.692423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:33.692423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:33.692423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:33.692423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.692423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:33.692423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:33.692423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:33.692423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.692675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:33.692675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:33.692675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:33.692675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:33.692675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:33.692675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:33.692675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:33.692675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:33.692675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.692675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:33.692675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:33.692675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.692675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:33.692922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.692922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:33.693447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:33.693752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:33.694095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:33.694095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:33.694095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:35.685514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:35.688045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:35.688045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:35.688045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:35.942030 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:35.942347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:35.943798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:35.943798 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:35.943798 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:35.943798 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:35.943798 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:35.943798 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:35.943798 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:35.943798 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:35.944829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.257 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:35.944829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:35.944829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:35.944829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:35.944829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:35.944829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:35.944829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:35.944829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:35.944829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:35.944829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:35.944829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:35.944829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:35.945388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:35.945388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:35.945388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:35.945388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:35.945388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:35.945388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:35.945388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:35.945388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:35.945388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:35.945388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:35.945388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:35.945388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:35.945388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:35.945889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:35.946504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:35.947313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:35.947313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:35.947313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:35.947313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:35.947313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:35.947313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:38.671801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:38.672437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:38.672590 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:38.673581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:38.674875 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:38.674875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.675121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:38.675121 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.675121 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:38.675121 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:38.675121 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.675121 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:38.675121 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.675121 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:38.676645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.676645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:38.676645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:38.676645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:38.676645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:38.676645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:38.676645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:38.676645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.676645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:38.676645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:38.676645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:38.676645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.677149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:38.677149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:38.677149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:38.677149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:38.677149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:38.677149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:38.677149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:38.677149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:38.677149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.677149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:38.677149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:38.677149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.677149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:38.677823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.677823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:38.681250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:38.682056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.683464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:38.683464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.683464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:38.683464 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:38.683464 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:38.683464 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:38.683464 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:36:38.685977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:38.685977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.686192 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:38.686647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:38.686647 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.686647 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:38.686647 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:38.686647 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.686647 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:38.686647 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.686647 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:38.690262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:38.691125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:38.691677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:38.691677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:38.691677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:36:38.691677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:38.691677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:38.691677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.691677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:38.691677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:38.691677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:38.691677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.691677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:38.691677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:38.691677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:38.692105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:38.692105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:38.692105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:38.692105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:38.692105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:38.692105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.692105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:38.692105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:38.692105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.692105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:38.692105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.692105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:38.692450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:38.692897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:38.693395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.693395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:38.693395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:38.693395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.693395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:38.693395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.693395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:38.693395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:38.693395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:38.693395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:38.695533 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:38.696795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:38.696795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.696960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:38.696960 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.696960 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:38.696960 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:38.696960 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.696960 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:38.696960 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.696960 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:38.697787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:38.698327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:38.698327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:38.698327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.698327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:38.698327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:38.698327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.698327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:38.698327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.698327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:38.698925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.698925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:38.698925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:38.698925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:38.698925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:38.698925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:38.698925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:38.698925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:38.698925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:38.698925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.698925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:38.699336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:38.699671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.699671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:38.699671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.699671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:38.700402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:38.701720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:38.702283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.702283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:38.702283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.702283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:38.702283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:38.702283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:38.702283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:38.702283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:38.702621 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:38.702621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.702621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:38.702621 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.702621 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:38.702621 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:38.702621 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.702621 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:38.702621 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.702621 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:38.704254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.704254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:38.704254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:38.704254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:38.704254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:38.704254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:38.704254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:38.704254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.704254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:38.704254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:38.704254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:38.704254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.706303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:38.706303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:38.706303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:38.706303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:38.706303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:38.706303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:38.706303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:38.706303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:38.706303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.706303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:38.706303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:38.706303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.706303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:38.706781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:38.707276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:38.707792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:38.707792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:38.707792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:38.707792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:38.707792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:38.707792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:40.686462 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:40.687348 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:40.687348 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:40.688391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:40.725049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:40.725605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:40.726383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:40.726383 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:40.726383 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:40.726383 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:40.726383 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:40.726383 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:40.726383 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:40.726383 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:40.727395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:40.727395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:40.727395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:40.727395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:40.727395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:40.727395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:40.727395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:40.727395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:40.727395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:40.727395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:40.727395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:40.727395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:40.728728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:40.728728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:40.728728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:40.728728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:40.728728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:40.728728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:40.728728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:40.728728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:40.728728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:40.728728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:40.728728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:40.728728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:40.728728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:40.729693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:40.730827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:40.732495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:40.732495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:40.732495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:40.732495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:40.732495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:40.732495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:43.672682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:43.674354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:43.674354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:43.674802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:43.675138 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:43.675412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.676361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:43.676361 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.676361 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:43.676361 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:43.676361 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.676361 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:43.676361 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.676361 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:43.678286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.678286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:43.678286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:43.678286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:43.678286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:43.678286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:43.678286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:43.678286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.678286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:43.678286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:43.678286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:43.678286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.678665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:43.678665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:43.678665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:43.678665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:43.678665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:43.678665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:43.678665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:43.678665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:43.678665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.678665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:43.678665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:43.678665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.678665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:43.679012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.679012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:43.679909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:43.680408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.680923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:43.680923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.680923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:43.680923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:43.683230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:43.683230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:43.683230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:36:43.683230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:43.683230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.683230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:43.683230 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.683230 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:43.683230 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:43.683585 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.683585 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:43.683585 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.683585 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:43.683717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:43.684247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:43.684247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:43.684247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.684247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:43.684247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:43.684247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.684247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:43.684247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.684247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:43.684540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.684540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:43.684540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:43.684540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:43.684540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:43.684540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:43.684540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:43.684540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:36:43.684540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:43.684540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:43.684540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.684540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:43.684899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:43.685330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.685330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:43.685330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.685330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:43.685803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:43.686308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:43.686827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.686827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:43.686827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:43.686827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.686827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:43.686827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.686827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:43.690932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:43.691191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:43.691323 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:43.691447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:43.693741 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:43.694100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.694743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:43.694743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.694743 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:43.694743 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:43.694743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.694743 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:43.694743 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.694743 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:43.695554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:43.695997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:43.695997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:43.695997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.695997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:43.695997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:43.695997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.695997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:43.695997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.695997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:43.696517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.696517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:43.696517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:43.696517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:43.696517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:43.696517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:43.696517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:43.696517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:43.696517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:43.696517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.696517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:43.696872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:43.697303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.697303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:43.697303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.697303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:43.697672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:43.698187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:43.698716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.698716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:43.698716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.698716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:43.700141 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:43.700410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:43.700582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:43.700742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:43.700976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:43.701135 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:43.701264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.701669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:43.701669 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.701669 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:43.701669 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:43.701669 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.701669 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:43.701669 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.701669 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:43.702573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.702573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:43.702573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:43.702573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:43.702573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:43.702573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:43.702573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:43.702573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.702573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:43.702573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:43.702573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:43.702573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.702972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:43.702972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:43.702972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:43.702972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:43.702972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:43.702972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:43.702972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:43.702972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:43.702972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.702972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:43.702972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:43.702972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.702972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:43.703348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.703348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:43.703603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:43.704080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:43.704639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:43.704639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:43.704639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:45.686980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:45.687716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:45.687716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:45.688695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:45.690366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=490,neutron:UPDATE=2 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:36:45.722172 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:45.722598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:45.723543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:45.723543 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:45.723543 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:45.723543 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:45.723543 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:45.723543 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:45.723543 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:45.723543 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:45.724622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:45.724622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:45.724622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:45.724622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:45.724622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:45.724622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:45.724622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:45.724622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:45.724622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:45.724622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:45.724622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:45.724622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:45.725359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:45.725359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:45.725359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:45.725359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:45.725359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:45.725359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:45.725359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:45.725359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:45.725359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:45.725359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:45.725359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:45.725359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:45.725359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:45.726057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:45.726876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:45.727870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:45.727870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:45.727870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:45.727870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:45.727870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:45.727870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:48.673709 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:48.674641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:48.674789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:48.676095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:48.676523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:48.676770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.677975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:48.677975 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.677975 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:48.677975 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:48.677975 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.677975 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:48.677975 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.677975 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:48.680435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.680435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:48.680435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:48.680435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:48.680435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:48.680435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:48.680435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:48.680435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.680435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:48.680435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:48.680435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:48.680435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.680759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:48.680759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:48.680759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:48.680759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:48.680759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:48.680759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:48.680759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:48.680759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:48.680759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.680759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:48.680759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:48.680759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.680759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:48.681099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.681099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:48.682583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:48.683023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.683502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:48.683502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.683502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:48.683502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:48.683502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:48.683646 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:48.683802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:36:48.684429 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:48.684716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.685460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:48.685460 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.685460 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:48.685460 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:48.685460 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.685460 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:48.685460 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.685460 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:48.686628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:48.687021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:48.687021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:48.687021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.687021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:48.687021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:48.687021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.687021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:48.687021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.687021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:48.687986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.687986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:48.687986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:48.687986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:48.687986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:48.687986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:48.687986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:48.687986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:36:48.687986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:48.687986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:48.687986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.687986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:48.688334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:48.688688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.688688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:48.688688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.688688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:48.689728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:48.690133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:48.690666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.690666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:48.690666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:48.690666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.690666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:48.690666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.690666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:48.691423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:48.691783 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:48.691889 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:48.692034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:48.693240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:48.693605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.694294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:48.694294 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.694294 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:48.694294 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:48.694294 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.694294 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:48.694294 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.694294 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:48.694998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:48.695403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:48.695403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:48.695403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.695403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:48.695403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:48.695403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.695403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:48.695403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.695403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:48.695965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.695965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:48.695965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:48.695965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:48.695965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:48.695965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:48.695965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:48.695965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:48.695965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:48.695965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.695965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:48.696314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:48.696659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.696659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:48.696659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.696659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:48.697007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:48.697450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:48.697922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.697922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:48.697922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.697922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:48.700722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:48.701100 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:48.701215 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:48.701385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:48.701836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:48.702168 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:48.702444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.703090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:48.703090 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.703090 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:48.703090 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:48.703090 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.703090 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:48.703090 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.703090 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:48.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:48.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:48.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:48.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:48.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:48.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:48.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:48.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:48.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:48.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.704295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:48.704295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:48.704295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:48.704295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:48.704295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:48.704295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:48.704295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:48.704295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:48.704295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.704295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:48.704295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:48.704295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.704295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:48.704619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.704619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:48.704892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:48.705380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:48.705865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:48.705865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:48.705865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:50.687914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:50.688681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:50.688910 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:50.690423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:50.722911 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:50.723371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:50.724331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:50.724331 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:50.724331 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:50.724331 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:50.724331 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:50.724331 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:50.724331 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:50.724331 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:50.725356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:50.725356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:50.725356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:50.725356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:50.725356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:50.725356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:50.725356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:50.725356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:50.725356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:50.725356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:50.725356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:50.725356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:50.726286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:50.726286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:50.726286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:50.726286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:50.726286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:50.726286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:50.726286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:50.726286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:50.726286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:50.726286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:50.726286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:50.726286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:50.726286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:50.726997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:50.727612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:50.728287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:50.728287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:50.728287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:50.728287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:50.728287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:50.728287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:53.674662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:53.675750 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:53.676003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:53.677089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:53.677317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:53.677442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.678027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:53.678027 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.678027 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:53.678027 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:53.678027 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.678027 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:53.678027 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.678027 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:53.679744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.679744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:53.679744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:53.679744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:53.679744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:53.679744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:53.679744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:53.679744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.679744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:53.679744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:53.679744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:53.679744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.680336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:53.680336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:53.680336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:53.680336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:53.680336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:53.680336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:53.680336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:53.680336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:53.680336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.680336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:53.680336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:53.680336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.680336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:53.680835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.680835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:53.681717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:53.682468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.683358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:53.683358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.683358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:53.683358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:53.683686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:53.683803 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:53.683922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:36:53.684196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:53.684359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:53.684722 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.684722 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:53.684722 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:53.684722 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.684722 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:53.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.684722 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:53.685455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:53.685881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:53.685881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:53.685881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.685881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:53.685881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:53.685881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.685881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:53.685881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.685881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:53.686255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.686255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:53.686255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:53.686255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:53.686255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:53.686255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:53.686255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:53.686255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:36:53.686255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:53.686255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:53.686255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.686255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:53.686600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:53.686998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.686998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:53.686998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.686998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:53.687307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:53.687951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:53.688656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.688656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:53.688656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:53.688656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.688656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:53.688656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.688656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:53.691909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:53.692245 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:53.692371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:53.692498 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:53.692758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:53.692908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.693301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:53.693301 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.693301 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:53.693301 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:53.693301 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.693301 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:53.693301 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.693301 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:53.694026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:53.694589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:53.694589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:53.694589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.694589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:53.694589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:53.694589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.694589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:53.694589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.694589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:53.694957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.694957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:53.694957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:53.694957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:53.694957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:53.694957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:53.694957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:53.694957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:53.694957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:53.694957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.694957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:53.695437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:53.695962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:53.696542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:53.697311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:53.697311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:53.697311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:53.697311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:53.697311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.697311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:53.697311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:53.697311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.697311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:53.697311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.697311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:53.701289 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:53.701613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:53.701735 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:53.701888 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:53.703772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:53.703996 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:53.704118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.704608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:53.704608 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.704608 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:53.704608 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:53.704608 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.704608 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:53.704608 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.704608 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:53.705424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.705424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:53.705424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:53.705424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:53.705424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:53.705424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:53.705424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:53.705424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.705424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:53.705424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:53.705424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:53.705424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.705901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:53.705901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:53.705901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:53.705901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:53.705901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:53.705901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:53.705901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:53.705901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:53.705901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.705901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:53.705901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:53.705901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.705901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:53.706427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:53.706980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:53.707709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:53.707709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:53.707709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:53.707709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:53.707709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:53.707709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:55.688609 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:55.689377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:55.689377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:55.690428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:55.726699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:55.726861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:55.727415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:55.727415 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:55.727415 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:55.727415 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:55.727415 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:55.727415 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:55.727415 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:55.727415 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:55.728420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:55.728420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:55.728420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:55.728420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:55.728420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:55.728420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:55.728420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:55.728420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:55.728420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:55.728420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:55.728420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:55.728420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:55.729197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:55.729197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:55.729197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:55.729197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:55.729197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:55.729197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:55.729197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:55.729197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:55.729197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:55.729197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:55.729197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:55.729197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:55.729197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:55.729818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:55.730478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:55.731384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:55.731384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:55.731384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:55.731384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:55.731384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:55.731384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:58.675389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:58.676187 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:58.676331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:58.677156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:58.677315 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:58.677472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.678600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:58.678600 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.678600 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:58.678600 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:58.678600 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.678600 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:58.678600 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.678600 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:58.679664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.679664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:58.679664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:58.679664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:58.679664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:58.679664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:58.679664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:58.679664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.679664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:58.679664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:58.679664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:58.679664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.680353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:58.680353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:58.680353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:58.680353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:58.680353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:58.680353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:58.680353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:58.680353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:58.680353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.680353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:58.680353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:58.680353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.680353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:58.680879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:58.681543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.682367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:58.682367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:58.682367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.682367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:58.682367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.682367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:58.683848 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:58.687800 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:58.687875 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:58.687963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:36:58.688498 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:58.688652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.689027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:58.689027 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.689027 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:58.689027 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:58.689027 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.689027 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:58.689027 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.689027 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:58.689706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:58.690369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:58.690934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:58.690934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:58.690934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:36:58.690934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:58.690934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:58.690934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.690934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:58.690934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:58.690934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:58.690934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.690934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:58.690934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:58.690934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:58.691491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:58.691491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:58.691491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:58.691491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:58.691491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:58.691491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.691491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:58.691491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:58.691491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.691491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:58.691491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.691491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:58.691989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:58.692939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:58.693649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.693649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:58.693649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:58.693649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.693649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:58.693649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.693649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:58.693649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:58.693649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:58.693649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:58.694184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:58.694184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:58.694184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.694184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:58.694184 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.694184 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:58.694184 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:58.694184 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.694184 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:58.694184 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.694184 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:58.694184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:36:58.694734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.695492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:36:58.695492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:36:58.695492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.695492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:36:58.695492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.695492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:36:58.695492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.695492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:58.695492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:58.695492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:58.695492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:58.695492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:58.695492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:58.696042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:58.696042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:58.696042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.696042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:58.696042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:58.696042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:58.696042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.696042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:58.696042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:58.696042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:58.696042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:58.696042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:58.696042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:58.696590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:58.696590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:58.696590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.696590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:58.696590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:58.696590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.696590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:58.696590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.696590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:58.696590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.696590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:58.696590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.697064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.697807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:36:58.703573 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:36:58.704117 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:36:58.704341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:36:58.704544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:36:58.704947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:58.705298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:36:58.705484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.706099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:36:58.706099 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.706099 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:36:58.706099 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:36:58.706099 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.706099 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:36:58.706099 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.706099 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:36:58.707155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.707155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:36:58.707155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:58.707155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:36:58.707155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:36:58.707155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:58.707155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:58.707155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.707155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:58.707155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:58.707155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:36:58.707155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.707552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:36:58.707552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:58.707552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:36:58.707552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:58.707552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:36:58.707552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:36:58.707552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:58.707552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:36:58.707552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.707552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:36:58.707552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:36:58.707552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.707552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:36:58.708322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.708322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:36:58.708508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:36:58.708950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:36:58.709458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:36:58.709458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:36:58.709458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:00.689120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:00.690000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:00.690076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:00.691652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:00.730258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:00.730491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:00.731070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:00.731070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:00.731070 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:00.731070 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:00.731070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:00.731070 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:00.731070 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:00.731070 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:00.731792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:00.731792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:00.731792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:00.731792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:00.731792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:00.731792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:00.731792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:00.731792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:00.731792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:00.731792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:00.731792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:00.731792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:00.732461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:00.732461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:00.732461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:00.732461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:00.732461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:00.732461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:00.732461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:00.732461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:00.732461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:00.732461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:00.732461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:00.732461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:00.732461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:00.733041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:00.733796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:00.734682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:00.734682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:00.734682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:00.734682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:00.734682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:00.734682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:03.676587 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:03.677415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:03.677695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:03.679453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:03.679871 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:03.680043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.680901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:03.680901 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.680901 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:03.680901 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:03.680901 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.680901 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:03.680901 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.680901 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:03.681733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.681733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:03.681733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:03.681733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:03.681733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:03.681733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:03.681733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:03.681733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.681733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:03.681733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:03.681733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:03.681733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.682106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:03.682106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:03.682106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:03.682106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:03.682106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:03.682106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:03.682106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:03.682106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:03.682106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.682106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:03.682106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:03.682106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.682106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:03.682550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.682550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:03.682785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:03.683252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.683796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:03.683796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.683796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:03.686853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:03.687230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:03.687349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:03.687472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:37:03.687873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:03.688063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.688502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:03.688502 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.688502 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:03.688502 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:03.688502 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.688502 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:03.688502 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.688502 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:03.689405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:03.689962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:03.689962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:03.689962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.689962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:03.689962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:03.689962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.689962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:03.689962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.689962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:03.690447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.690447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:03.690447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:03.690447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:03.690447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:03.690447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:03.690447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:03.690447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:37:03.690447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:03.690447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:03.690447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.690447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:03.690980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:03.691684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:03.692158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:03.692727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:03.692727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:03.692727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:03.692727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:03.692727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:03.692727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.692727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:03.692727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:03.692727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.692727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:03.692727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.692727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:03.693070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:03.693113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:03.693374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:03.693374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:03.693993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:03.693993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.694220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:03.694220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.694220 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:03.694220 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:03.694220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.694220 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:03.694220 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.694220 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:03.695064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:03.695643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:03.695643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:03.695643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.695643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:03.695643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:03.695643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.695643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:03.695643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.695643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:03.696053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.696053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:03.696053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:03.696053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:03.696053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:03.696053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:03.696053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:03.696053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:03.696053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:03.696053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.696053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:03.696509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:03.696918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.696918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:03.696918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.696918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:03.697031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:03.697515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:03.698849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.698849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:03.698849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.698849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:03.704105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:03.704426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:03.704539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:03.704692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:03.704922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:03.705134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:03.705295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.705723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:03.705723 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.705723 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:03.705723 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:03.705723 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.705723 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:03.705723 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.705723 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:03.706502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.706502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:03.706502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:03.706502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:03.706502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:03.706502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:03.706502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:03.706502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.706502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:03.706502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:03.706502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:03.706502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.706777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:03.706777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:03.706777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:03.706777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:03.706777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:03.706777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:03.706777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:03.706777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:03.706777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.706777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:03.706777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:03.706777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.706777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:03.707031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.707031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:03.707453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:03.707799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:03.708191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:03.708191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:03.708191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:05.689835 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:05.690613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:05.690732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:05.691820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:05.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:05.743728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:05.744548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:05.744548 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:05.744548 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:05.744548 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:05.744548 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:05.744548 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:05.744548 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:05.744548 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:05.745442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:05.745442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:05.745442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:05.745442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:05.745442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:05.745442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:05.745442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:05.745442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:05.745442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:05.745442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:05.745442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:05.745442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:05.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:05.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:05.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:05.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:05.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:05.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:05.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:05.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:05.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:05.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:05.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:05.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:05.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:05.746082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:05.746082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:05.746448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:05.746795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:05.747197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:05.747197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:05.747197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:08.677204 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:08.678282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:08.678586 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:08.679740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:08.680141 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:08.680473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.681503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:08.681503 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.681503 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:08.681503 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:08.681503 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.681503 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:08.681503 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.681503 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:08.683267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.683267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:08.683267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:08.683267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:08.683267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:08.683267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:08.683267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:08.683267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.683267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:08.683267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:08.683267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:08.683267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.683674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:08.683674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:08.683674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:08.683674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:08.683674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:08.683674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:08.683674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:08.683674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:08.683674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.683674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:08.683674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:08.683674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.683674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:08.683932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.683932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:08.685388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:08.685763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.686109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:08.686109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.686109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:08.687259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:08.687413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:08.687510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:08.687617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:37:08.687838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:08.687969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.688351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:08.688351 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.688351 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:08.688351 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:08.688351 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.688351 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:08.688351 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.688351 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:08.689379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:08.690288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:08.690920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:08.690920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:08.690920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:37:08.690920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:08.690920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:08.690920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.690920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:08.690920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:08.690920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:08.690920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.690920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:08.690920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:08.690920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:08.691501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:08.691501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:08.691501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:08.691501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:08.691501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:08.691501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.691501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:08.691501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:08.691501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.691501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:08.691501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.691501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:08.692015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:08.692802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:08.693639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.693639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:08.693639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:08.693639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.693639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:08.693639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.693639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:08.693639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:08.693639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:08.694079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:08.694079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:08.694079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:08.694291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.694510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:08.694510 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.694510 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:08.694510 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:08.694510 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.694510 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:08.694510 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.694510 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:08.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:08.695518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:08.695518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:08.695518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.695518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:08.695518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:08.695518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.695518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:08.695518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.695518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:08.695932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.695932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:08.695932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:08.695932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:08.695932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:08.695932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:08.695932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:08.695932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:08.695932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:08.695932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.695932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:08.696264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:08.696633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.696633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:08.696633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.696633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:08.697149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:08.697624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:08.698091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.698091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:08.698091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.698091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:08.704489 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:08.704804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:08.704903 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:08.705064 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:08.705407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:08.705657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:08.705802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.706291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:08.706291 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.706291 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:08.706291 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:08.706291 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.706291 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:08.706291 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.706291 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:08.707094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.707094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:08.707094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:08.707094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:08.707094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:08.707094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:08.707094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:08.707094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.707094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:08.707094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:08.707094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:08.707094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.707467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:08.707467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:08.707467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:08.707467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:08.707467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:08.707467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:08.707467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:08.707467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:08.707467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.707467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:08.707467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:08.707467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.707467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:08.707822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.707822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:08.708222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:08.708740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:08.709281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:08.709281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:08.709281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:10.690761 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:10.691653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:10.691653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:10.692707 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:10.729031 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:10.729633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:10.730452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:10.730452 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:10.730452 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:10.730452 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:10.730452 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:10.730452 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:10.730452 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:10.730452 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:10.731562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:10.731562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:10.731562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:10.731562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:10.731562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:10.731562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:10.731562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:10.731562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:10.731562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:10.731562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:10.731562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:10.731562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:10.732334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:10.732334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:10.732334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:10.732334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:10.732334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:10.732334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:10.732334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:10.732334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:10.732334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:10.732334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:10.732334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:10.732334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:10.732334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:10.733080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:10.734005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:10.735163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:10.735163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:10.735163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:10.735163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:10.735163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:10.735163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:13.678239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:13.679029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:13.679029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:13.680056 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:13.680410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:13.680704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.681604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:13.681604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.681604 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:13.681604 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:13.681604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.681604 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:13.681604 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.681604 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:13.683143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.683143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:13.683143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:13.683143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:13.683143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:13.683143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:13.683143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:13.683143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.683143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:13.683143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:13.683143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:13.683143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.683870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:13.683870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:13.683870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:13.683870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:13.683870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:13.683870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:13.683870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:13.683870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:13.683870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.683870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:13.683870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:13.683870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.683870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:13.684620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.684620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:13.684884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:13.685509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.686022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:13.686022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.686022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:13.687571 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:13.687841 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:13.687967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:13.688109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:37:13.688532 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:13.688741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.689251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:13.689251 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.689251 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:13.689251 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:13.689251 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.689251 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:13.689251 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.689251 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:13.689983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:13.690451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:13.690451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:13.690451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.690451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:13.690451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:13.690451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.690451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:13.690451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.690451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:13.690915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.690915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:13.690915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:13.690915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:13.690915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:13.690915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:13.690915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:13.690915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:37:13.690915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:13.690915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:13.690915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.690915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:13.691288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:13.691678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.691678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:13.691678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.691678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:13.691992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:13.692462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:13.692977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.692977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:13.692977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:13.692977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.692977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:13.692977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.692977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:13.693796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:13.694061 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:13.694177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:13.694324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:13.694683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:13.694925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.695355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:13.695355 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.695355 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:13.695355 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:13.695355 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.695355 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:13.695355 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.695355 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:13.695872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:13.696312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:13.696312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:13.696312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.696312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:13.696312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:13.696312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.696312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:13.696312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.696312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:13.696766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.696766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:13.696766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:13.696766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:13.696766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:13.696766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:13.696766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:13.696766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:13.696766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:13.696766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.696766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:13.697108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:13.697595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.697595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:13.697595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.697595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:13.698880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:13.699410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:13.699909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.699909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:13.699909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.699909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:13.704866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:13.705236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:13.705428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:13.705591 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:13.705959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:13.706340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:13.706552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.707243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:13.707243 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.707243 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:13.707243 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:13.707243 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.707243 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:13.707243 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.707243 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:13.708109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.708109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:13.708109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:13.708109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:13.708109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:13.708109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:13.708109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:13.708109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.708109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:13.708109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:13.708109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:13.708109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.708628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:13.708628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:13.708628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:13.708628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:13.708628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:13.708628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:13.708628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:13.708628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:13.708628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.708628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:13.708628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:13.708628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.708628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:13.709091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:13.709605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:13.710336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:13.710336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:13.710336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:13.710336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:13.710336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:13.710336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:15.691165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:15.692108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:15.692108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:15.693152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:15.730438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:15.730791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:15.731781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:15.731781 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:15.731781 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:15.731781 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:15.731781 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:15.731781 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:15.731781 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:15.731781 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:15.732969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:15.732969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:15.732969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:15.732969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:15.732969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:15.732969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:15.732969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:15.732969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:15.732969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:15.732969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:15.732969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:15.732969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:15.733490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:15.733490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:15.733490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:15.733490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:15.733490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:15.733490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:15.733490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:15.733490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:15.733490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:15.733490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:15.733490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:15.733490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:15.733490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:15.733869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:15.733869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:15.734052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:15.734581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:15.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:15.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:15.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:18.678900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:18.679459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:18.679459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:18.680422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:18.680487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:18.681119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.681654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:18.681654 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.681654 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:18.681654 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:18.681654 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.681654 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:18.681654 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.681654 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:18.682945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.682945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:18.682945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:18.682945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:18.682945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:18.682945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:18.682945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:18.682945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.682945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:18.682945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:18.682945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:18.682945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.683470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:18.683470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:18.683470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:18.683470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:18.683470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:18.683470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:18.683470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:18.683470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:18.683470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.683470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:18.683470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:18.683470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.683470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:18.683901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.683901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:18.683992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:18.684479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.685049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:18.685049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.685049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:18.687988 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:18.688296 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:18.688419 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:18.688541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:37:18.689123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:18.689571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.690913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:18.690913 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.690913 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:18.690913 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:18.690913 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.690913 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:18.690913 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.690913 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:18.691167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:18.691680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:18.691680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:18.691680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.691680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:18.691680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:18.691680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.691680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:18.691680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.691680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:18.693677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.693677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:18.693677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:18.693677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:18.693677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:18.693677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:18.693677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:18.693677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:37:18.693677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:18.693677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:18.693677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.693677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:18.694131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:18.694560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.694560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:18.694560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.694560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:18.694995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:18.695487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:18.696018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.696018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:18.696018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:18.696018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.696018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:18.696018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.696018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:18.696018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:18.696018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:18.696018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:18.696391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:18.696391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:18.696391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.696712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:18.696712 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.696712 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:18.696712 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:18.696712 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.696712 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:18.696712 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.696712 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:18.697431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:18.697981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:18.697981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:18.697981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.697981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:18.697981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:18.697981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.697981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:18.697981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.697981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:18.698461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.698461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:18.698461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:18.698461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:18.698461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:18.698461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:18.698461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:18.698461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:18.698461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:18.698461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.698461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:18.698874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:18.699279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.699279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:18.699279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.699279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:18.699564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:18.700242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:18.700968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.700968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:18.700968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.700968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:18.705261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:18.705484 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:18.705592 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:18.705738 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:18.706047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:18.706370 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:18.706515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.706942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:18.706942 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.706942 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:18.706942 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:18.706942 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.706942 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:18.706942 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.706942 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:18.707707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.707707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:18.707707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:18.707707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:18.707707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:18.707707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:18.707707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:18.707707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.707707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:18.707707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:18.707707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:18.707707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.708086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:18.708086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:18.708086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:18.708086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:18.708086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:18.708086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:18.708086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:18.708086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:18.708086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.708086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:18.708086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:18.708086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.708086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:18.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.708524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:18.708684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:18.709296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:18.709846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:18.709846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:18.709846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:20.692439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:20.692439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:20.693024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:20.693779 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:20.745238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:20.745449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:20.746908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:20.746908 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:20.746908 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:20.746908 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:20.746908 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:20.746908 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:20.746908 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:20.746908 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:20.749138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:20.749138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:20.749138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:20.749138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:20.749138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:20.749138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:20.749138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:20.749138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:20.749138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:20.749138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:20.749138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:20.749138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:20.749536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:20.749536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:20.749536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:20.749536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:20.749536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:20.749536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:20.749536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:20.749536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:20.749536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:20.749536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:20.749536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:20.749536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:20.749536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:20.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:20.750296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:20.750793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:20.750793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:20.750793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:20.750793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:20.750793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:20.750793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:23.679689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:23.680535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:23.680535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:23.681760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:23.682097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:23.682379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.683342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:23.683342 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.683342 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:23.683342 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:23.683342 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.683342 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:23.683342 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.683342 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:23.684999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.684999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:23.684999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:23.684999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:23.684999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:23.684999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:23.684999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:23.684999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.684999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:23.684999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:23.684999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:23.684999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.685747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:23.685747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:23.685747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:23.685747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:23.685747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:23.685747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:23.685747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:23.685747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:23.685747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.685747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:23.685747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:23.685747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.685747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:23.686307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.686307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:23.686973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:23.687548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.688079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:23.688079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.688079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:23.688386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:23.688754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:23.688854 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:23.688980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:37:23.689562 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:23.689941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.690762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:23.690762 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.690762 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:23.690762 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:23.690762 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.690762 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:23.690762 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.690762 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:23.691498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:23.692020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:23.692020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:23.692020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.692020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:23.692020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:23.692020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.692020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:23.692020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.692020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:23.692376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.692376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:23.692376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:23.692376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:23.692376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:23.692376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:23.692376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:23.692376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:37:23.692376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:23.692376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:23.692376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.692376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:23.692701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:23.693329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.693329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:23.693329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.693329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:23.693969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:23.694599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:23.695110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.695110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:23.695110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:23.695110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.695110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:23.695110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.695110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:23.695603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:23.695653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:23.695753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:23.695870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:23.696221 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:23.696521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.696977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:23.696977 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.696977 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:23.696977 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:23.696977 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.696977 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:23.696977 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.696977 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:23.697728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:23.698150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:23.698150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:23.698150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.698150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:23.698150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:23.698150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.698150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:23.698150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.698150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:23.698614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.698614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:23.698614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:23.698614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:23.698614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:23.698614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:23.698614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:23.698614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:23.698614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:23.698614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.698614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:23.698969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:23.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:23.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:23.699684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:23.700219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:23.700721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.700721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:23.700721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.700721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:23.705610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:23.706023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:23.706330 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:23.706694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:23.707177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:23.707696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:23.707945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.708775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:23.708775 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.708775 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:23.708775 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:23.708775 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.708775 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:23.708775 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.708775 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:23.710583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.710583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:23.710583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:23.710583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:23.710583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:23.710583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:23.710583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:23.710583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.710583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:23.710583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:23.710583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:23.710583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.710958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:23.710958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:23.710958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:23.710958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:23.710958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:23.710958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:23.710958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:23.710958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:23.710958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.710958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:23.710958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:23.710958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.710958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:23.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:23.712978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:23.713848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:23.714625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:23.714625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:23.714625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:25.695510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:25.696320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:25.696397 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:25.697767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:25.729860 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:25.730022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:25.735023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:25.735509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:25.735904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:25.736498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:25.737389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:25.737389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:25.737389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:25.737389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:25.737389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:25.737389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:25.737389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:25.737389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:25.737389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:28.680936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:28.681544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:28.681544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:28.682585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:28.683240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:28.683755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.685100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:28.685100 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.685100 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:28.685100 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:28.685100 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.685100 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:28.685100 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.685100 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:28.686536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.686536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:28.686536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:28.686536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:28.686536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:28.686536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:28.686536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:28.686536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.686536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:28.686536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:28.686536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:28.686536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.686939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:28.686939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:28.686939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:28.686939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:28.686939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:28.686939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:28.686939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:28.686939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:28.686939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.686939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:28.686939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:28.686939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.686939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:28.687320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.687320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:28.688074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:28.688555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.689059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:28.689059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.689059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:28.689059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:28.689483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:28.689648 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:28.689858 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:37:28.690325 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:28.690551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.691084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:28.691084 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.691084 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:28.691084 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:28.691084 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.691084 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:28.691084 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.691084 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:28.692889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:28.693387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:28.693793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:28.693793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:28.693793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:37:28.693793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:28.693793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:28.693793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.693793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:28.693793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:28.693793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:28.693793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.693793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:28.693793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:28.693793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:28.694170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:28.694170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:28.694170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:28.694170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:28.694170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:28.694170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.694170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:28.694170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:28.694170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.694170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:28.694170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.694170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:28.694512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:28.694956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:28.695479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.695479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:28.695479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:28.695479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.695479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:28.695479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.695479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:28.696483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:28.696483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:28.696483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:28.696483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:28.696777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:28.697078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:28.697681 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.697681 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:28.697681 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:28.697681 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.697681 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:28.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:28.698292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:28.698730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:28.698730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:28.698730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.698730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:28.698730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:28.698730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.698730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:28.698730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.698730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:28.700384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.700384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:28.700384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:28.700384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:28.700384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:28.700384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:28.700384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:28.700384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:28.700384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:28.700384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.700384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:28.701136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:28.701943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:28.702468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:28.703143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:28.703143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:28.703143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:28.703143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:28.703143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.703143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:28.703143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:28.703143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.703143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:28.703143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.703143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:28.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:28.706424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:28.706589 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:28.706858 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:28.707389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:28.707788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:28.708009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.708774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:28.708774 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.708774 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:28.708774 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:28.708774 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.708774 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:28.708774 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.708774 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:28.709886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.709886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:28.709886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:28.709886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:28.709886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:28.709886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:28.709886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:28.709886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.709886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:28.709886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:28.709886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:28.709886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.710314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:28.710314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:28.710314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:28.710314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:28.710314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:28.710314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:28.710314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:28.710314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:28.710314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.710314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:28.710314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:28.710314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.710314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:28.710697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.710697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:28.711175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:28.711633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:28.712019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:28.712019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:28.712019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:30.696366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:30.706011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:30.706011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:30.706011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:30.738461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:30.738925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:30.740041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:30.740041 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:30.740041 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:30.740041 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:30.740041 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:30.740041 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:30.740041 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:30.740041 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:30.741295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:30.741295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:30.741295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:30.741295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:30.741295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:30.741295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:30.741295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:30.741295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:30.741295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:30.741295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:30.741295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:30.741295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:30.741813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:30.741813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:30.741813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:30.741813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:30.741813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:30.741813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:30.741813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:30.741813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:30.741813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:30.741813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:30.741813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:30.741813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:30.741813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:30.742415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:30.743057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:30.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:30.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:30.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:30.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:30.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:30.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:33.681588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:33.682510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:33.682510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:33.683395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:33.683704 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:33.683983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.684864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:33.684864 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.684864 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:33.684864 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:33.684864 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.684864 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:33.684864 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.684864 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:33.686529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.686529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:33.686529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:33.686529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:33.686529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:33.686529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:33.686529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:33.686529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.686529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:33.686529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:33.686529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:33.686529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.687185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:33.687185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:33.687185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:33.687185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:33.687185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:33.687185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:33.687185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:33.687185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:33.687185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.687185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:33.687185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:33.687185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.687185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:33.687800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.687800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:33.688127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:33.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.689660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:33.689660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.689660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:33.689660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:33.690024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:33.690182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:33.690382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:37:33.690819 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:33.691097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.691831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:33.691831 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.691831 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:33.691831 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:33.691831 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.691831 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:33.691831 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.691831 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:33.692728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:33.693237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:33.693237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:33.693237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.693237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:33.693237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:33.693237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.693237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:33.693237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.693237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:33.693737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.693737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:33.693737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:33.693737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:33.693737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:33.693737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:33.693737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:33.693737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:37:33.693737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:33.693737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:33.693737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.693737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:33.694106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:33.694492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.694492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:33.694492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.694492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:33.695069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:33.695641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:33.696125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.696125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:33.696125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:33.696125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.696125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:33.696125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.696125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:33.696346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:33.696584 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:33.696682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:33.696816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:33.697218 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:33.697634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.698354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:33.698354 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.698354 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:33.698354 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:33.698354 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.698354 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:33.698354 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.698354 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:33.699094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:33.699539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:33.699539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:33.699539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.699539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:33.699539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:33.699539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.699539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:33.699539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.699539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:33.699975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.699975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:33.699975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:33.699975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:33.699975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:33.699975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:33.699975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:33.699975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:33.699975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:33.699975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.699975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:33.700342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:33.700727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.700727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:33.700727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.700727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:33.701046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:33.701558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:33.702051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.702051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:33.702051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.702051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:33.706694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:33.707004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:33.707149 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:33.707409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:33.707811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:33.708145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:33.708380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.709013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:33.709013 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.709013 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:33.709013 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:33.709013 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.709013 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:33.709013 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.709013 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:33.709963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.709963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:33.709963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:33.709963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:33.709963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:33.709963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:33.709963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:33.709963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.709963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:33.709963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:33.709963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:33.709963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.710358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:33.710358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:33.710358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:33.710358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:33.710358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:33.710358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:33.710358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:33.710358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:33.710358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.710358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:33.710358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:33.710358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.710358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:33.710693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.710693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:33.711430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:33.805182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:33.805628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:33.805628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:33.805628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:35.697123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:35.697989 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:35.697989 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:35.699171 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:35.735046 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:35.735502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:35.736412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:35.736412 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:35.736412 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:35.736412 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:35.736412 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:35.736412 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:35.736412 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:35.736412 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:35.737448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:35.737448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:35.737448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:35.737448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:35.737448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:35.737448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:35.737448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:35.737448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:35.737448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:35.737448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:35.737448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:35.737448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:35.738074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:35.738074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:35.738074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:35.738074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:35.738074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:35.738074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:35.738074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:35.738074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:35.738074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:35.738074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:35.738074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:35.738074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:35.738074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:35.738650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:35.739309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:35.740095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:35.740095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:35.740095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:35.740095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:35.740095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:35.740095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:38.682174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:38.682625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:38.682657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:38.683635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:38.683934 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:38.684175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.684951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:38.684951 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.684951 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:38.684951 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:38.684951 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.684951 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:38.684951 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.684951 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:38.686127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.686127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:38.686127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:38.686127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:38.686127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:38.686127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:38.686127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:38.686127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.686127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:38.686127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:38.686127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:38.686127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.686432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:38.686432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:38.686432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:38.686432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:38.686432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:38.686432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:38.686432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:38.686432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:38.686432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.686432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:38.686432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:38.686432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.686432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:38.686668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.686668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:38.687323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:38.688104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.688801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:38.688801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.688801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:38.690143 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:38.690421 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:38.690464 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:38.690590 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:37:38.690900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:38.691096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.691612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:38.691612 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.691612 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:38.691612 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:38.691612 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.691612 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:38.691612 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.691612 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:38.692455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:38.692767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:38.692767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:38.692767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.692767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:38.692767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:38.692767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.692767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:38.692767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.692767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:38.693413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.693413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:38.693413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:38.693413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:38.693413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:38.693413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:38.693413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:38.693413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:37:38.693413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:38.693413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:38.693413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.693413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:38.694380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:38.694722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:38.695012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:38.695357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:38.695357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:38.695357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:38.695357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:38.695357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:38.695357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.695357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:38.695357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:38.695357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.695357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:38.695357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.695357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:38.697228 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:38.697228 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:38.697301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:38.697427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:38.697894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:38.698280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.698985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:38.698985 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.698985 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:38.698985 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:38.698985 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.698985 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:38.698985 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.698985 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:38.699791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:38.700114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:38.700114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:38.700114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.700114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:38.700114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:38.700114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.700114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:38.700114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.700114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:38.700777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.700777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:38.700777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:38.700777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:38.700777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:38.700777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:38.700777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:38.700777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:38.700777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:38.700777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.700777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:38.701169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:38.701589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.701589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:38.701589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.701589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:38.702162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:38.702517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:38.702854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.702854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:38.702854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.702854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:38.707050 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:38.707415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:38.707604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:38.707846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:38.708303 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:38.708639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:38.708854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.709653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:38.709653 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.709653 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:38.709653 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:38.709653 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.709653 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:38.709653 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.709653 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:38.710969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.710969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:38.710969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:38.710969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:38.710969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:38.710969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:38.710969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:38.710969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.710969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:38.710969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:38.710969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:38.710969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:38.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:38.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:38.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:38.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:38.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:38.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:38.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:38.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:38.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:38.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.711328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:38.711691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.711691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:38.712757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:38.713249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:38.714135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:38.714135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:38.714135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:40.698188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:40.699089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:40.699089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:40.699411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:40.732624 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:40.732914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:40.733626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:40.733626 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:40.733626 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:40.733626 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:40.733626 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:40.733626 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:40.733626 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:40.733626 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:40.734713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:40.734713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:40.734713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:40.734713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:40.734713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:40.734713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:40.734713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:40.734713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:40.734713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:40.734713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:40.734713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:40.734713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:40.735745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:40.735745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:40.735745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:40.735745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:40.735745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:40.735745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:40.735745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:40.735745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:40.735745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:40.735745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:40.735745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:40.735745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:40.735745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:40.736134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:40.736654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:40.737232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:40.737232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:40.737232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:40.737232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:40.737232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:40.737232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:43.682788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:43.683619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:43.683619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:43.684763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:43.685155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:43.685405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.686407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:43.686407 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.686407 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:43.686407 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:43.686407 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.686407 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:43.686407 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.686407 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:43.688191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.688191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:43.688191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:43.688191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:43.688191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:43.688191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:43.688191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:43.688191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.688191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:43.688191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:43.688191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:43.688191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.688812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:43.688812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:43.688812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:43.688812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:43.688812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:43.688812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:43.688812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:43.688812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:43.688812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.688812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:43.688812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:43.688812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.688812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:43.689388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.689388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:43.690996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:43.691420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.691888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:43.691888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.691888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:43.692026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:43.692583 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:43.692863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:43.693160 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:37:43.693726 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:43.694008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.694800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:43.694800 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.694800 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:43.694800 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:43.694800 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.694800 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:43.694800 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.694800 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:43.695428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:43.695822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:43.695822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:43.695822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.695822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:43.695822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:43.695822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.695822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:43.695822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.695822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:43.696046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.696046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:43.696046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:43.696046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:43.696046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:43.696046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:43.696046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:43.696046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:37:43.696046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:43.696046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:43.696046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.696046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:43.696419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:43.696769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.696769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:43.696769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.696769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:43.696880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:43.697310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:43.697774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.697774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:43.697774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:43.697774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.697774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:43.697774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.697774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:43.697774 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:43.698228 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:43.698304 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:43.698410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:43.698867 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:43.699253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.699925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:43.699925 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.699925 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:43.699925 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:43.699925 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.699925 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:43.699925 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.699925 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:43.700866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:43.701309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:43.701674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:43.701674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:43.701674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:43.701674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:43.701674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.701674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:43.701674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:43.701674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:43.701674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.701674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:43.701674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:43.701674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:43.701674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:43.702039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:43.702039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:43.702039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:43.702039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:43.702039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.702039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:43.702039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:43.702039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.702039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:43.702039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.702039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:43.702039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:43.702394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.702904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:43.707382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:43.707702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:43.707820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:43.707969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:43.708300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:43.708563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:43.708763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.709422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:43.709422 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.709422 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:43.709422 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:43.709422 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.709422 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:43.709422 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.709422 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:43.710273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.710273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:43.710273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:43.710273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:43.710273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:43.710273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:43.710273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:43.710273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.710273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:43.710273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:43.710273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:43.710273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:43.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:43.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:43.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:43.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:43.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:43.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:43.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:43.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:43.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:43.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:43.710931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.710931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:43.711064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:43.711504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:43.711982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:43.711982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:43.711982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:45.698603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:45.699118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:45.699341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:45.700247 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:45.701949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:37:45.730584 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:45.730807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:45.731524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:45.731524 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:45.731524 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:45.731524 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:45.731524 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:45.731524 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:45.731524 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:45.731524 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:45.732296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:45.732296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:45.732296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:45.732296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:45.732296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:45.732296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:45.732296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:45.732296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:45.732296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:45.732296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:45.732296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:45.732296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:45.732699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:45.732699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:45.732699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:45.732699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:45.732699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:45.732699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:45.732699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:45.732699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:45.732699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:45.732699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:45.732699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:45.732699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:45.732699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:45.733092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:45.733092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:45.733346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:45.733740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:45.734331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:45.734331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:45.734331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:48.683361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:48.684385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:48.684385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:48.685545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:48.685894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:48.686111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.687017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:48.687017 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.687017 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:48.687017 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:48.687017 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.687017 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:48.687017 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.687017 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:48.688706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.688706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:48.688706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:48.688706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:48.688706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:48.688706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:48.688706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:48.688706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.688706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:48.688706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:48.688706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:48.688706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.689469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:48.689469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:48.689469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:48.689469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:48.689469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:48.689469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:48.689469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:48.689469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:48.689469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.689469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:48.689469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:48.689469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.689469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:48.690174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.690174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:48.691023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:48.692002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.693129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:48.693129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.693129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:48.693129 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:48.693129 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:48.693458 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:48.693598 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:37:48.694059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:48.694352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.695098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:48.695098 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.695098 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:48.695098 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:48.695098 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.695098 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:48.695098 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.695098 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:48.696283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:48.696737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:48.696737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:48.696737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.696737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:48.696737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:48.696737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.696737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:48.696737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.696737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:48.697360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.697360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:48.697360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:48.697360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:48.697360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:48.697360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:48.697360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:48.697360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:37:48.697360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:48.697360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:48.697360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.697360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:48.697719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:48.698095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.698095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:48.698095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.698095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:48.698780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:48.699223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:48.699721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.699721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:48.699721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:48.699721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.699721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:48.699721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.699721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:48.699721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:48.699721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:48.699989 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:48.700132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:48.700556 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:48.700879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.701547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:48.701547 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.701547 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:48.701547 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:48.701547 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.701547 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:48.701547 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.701547 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:48.702436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:48.702799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:48.702799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:48.702799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.702799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:48.702799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:48.702799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.702799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:48.702799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.702799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:48.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:48.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:48.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:48.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:48.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:48.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:48.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:48.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:48.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.703606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:48.703928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:48.704312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.704312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:48.704312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.704312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:48.704973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:48.705452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:48.705882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.705882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:48.705882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.705882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:48.707800 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:48.708082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:48.708198 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:48.708384 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:48.708771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:48.709081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:48.709311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.709952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:48.709952 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.709952 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:48.709952 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:48.709952 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.709952 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:48.709952 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.709952 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:48.710827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.710827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:48.710827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:48.710827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:48.710827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:48.710827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:48.710827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:48.710827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.710827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:48.710827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:48.710827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:48.710827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.711148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:48.711148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:48.711148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:48.711148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:48.711148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:48.711148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:48.711148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:48.711148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:48.711148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.711148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:48.711148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:48.711148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.711148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:48.711506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.711506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:48.711941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:48.712399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:48.712904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:48.712904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:48.712904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:50.699287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:50.700198 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:50.700198 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:50.701569 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:50.735775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:50.736162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:50.737461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:50.737461 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:50.737461 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:50.737461 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:50.737461 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:50.737461 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:50.737461 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:50.737461 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:50.738479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:50.738479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:50.738479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:50.738479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:50.738479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:50.738479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:50.738479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:50.738479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:50.738479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:50.738479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:50.738479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:50.738479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:50.739327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:50.739327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:50.739327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:50.739327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:50.739327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:50.739327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:50.739327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:50.739327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:50.739327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:50.739327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:50.739327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:50.739327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:50.739327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:50.739876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:50.740530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:50.741347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:50.741347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:50.741347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:50.741347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:50.741347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:50.741347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:53.684048 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:53.684877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:53.684877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:53.685704 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:53.685829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:53.686055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.686914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:53.686914 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.686914 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:53.686914 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:53.686914 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.686914 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:53.686914 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.686914 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:53.688472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.688472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:53.688472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:53.688472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:53.688472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:53.688472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:53.688472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:53.688472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.688472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:53.688472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:53.688472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:53.688472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.689001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:53.689001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:53.689001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:53.689001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:53.689001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:53.689001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:53.689001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:53.689001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:53.689001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.689001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:53.689001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:53.689001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.689001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:53.689517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.689517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:53.690099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:53.690784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.691372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:53.691372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.691372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:53.692886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:53.693144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:53.693297 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:53.693421 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:37:53.693818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:53.694081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.694906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:53.694906 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.694906 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:53.694906 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:53.694906 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.694906 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:53.694906 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.694906 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:53.695660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:53.696090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:53.696090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:53.696090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.696090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:53.696090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:53.696090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.696090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:53.696090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.696090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:53.696610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.696610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:53.696610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:53.696610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:53.696610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:53.696610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:53.696610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:53.696610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:37:53.696610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:53.696610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:53.696610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.696610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:53.696938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:53.697318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.697318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:53.697318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.697318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:53.697644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:53.698062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:53.698598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.698598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:53.698598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:53.698598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.698598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:53.698598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.698598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:53.699505 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:53.699736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:53.699851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:53.700036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:53.700479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:53.700814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.701533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:53.701533 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.701533 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:53.701533 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:53.701533 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.701533 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:53.701533 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.701533 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:53.702282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:53.702684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:53.702684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:53.702684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.702684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:53.702684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:53.702684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.702684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:53.702684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.702684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:53.703160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.703160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:53.703160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:53.703160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:53.703160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:53.703160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:53.703160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:53.703160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:53.703160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:53.703160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.703160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:53.703485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:53.703840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.703840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:53.703840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.703840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:53.704347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:53.704792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:53.705275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.705275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:53.705275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.705275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:53.708069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:53.708366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:53.708516 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:53.708712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:53.709090 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:53.709474 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:53.709714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.710465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:53.710465 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.710465 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:53.710465 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:53.710465 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.710465 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:53.710465 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.710465 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:53.711785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.711785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:53.711785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:53.711785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:53.711785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:53.711785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:53.711785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:53.711785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.711785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:53.711785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:53.711785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:53.711785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.712190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:53.712190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:53.712190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:53.712190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:53.712190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:53.712190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:53.712190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:53.712190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:53.712190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.712190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:53.712190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:53.712190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.712190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:53.712625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.712625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:53.712920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:53.713414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:53.713962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:53.713962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:53.713962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:55.700030 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:55.700838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:55.700838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:55.701955 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:55.736825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:55.737083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:55.737689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:55.737689 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:55.737689 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:55.737689 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:55.737689 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:55.737689 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:55.737689 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:55.737689 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:55.738573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:55.738573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:55.738573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:55.738573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:55.738573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:55.738573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:55.738573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:55.738573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:55.738573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:55.738573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:55.738573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:55.738573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:55.739073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:55.739073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:55.739073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:55.739073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:55.739073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:55.739073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:55.739073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:55.739073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:55.739073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:55.739073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:55.739073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:55.739073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:55.739073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:55.739662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:55.740301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:55.741078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:55.741078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:55.741078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:55.741078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:55.741078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:55.741078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:58.684718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:58.685439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:58.685583 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:58.686585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:58.686846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:58.687155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.687767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:58.687767 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.687767 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:58.687767 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:58.687767 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.687767 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:58.687767 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.687767 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:58.689353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.689353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:58.689353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:58.689353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:58.689353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:58.689353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:58.689353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:58.689353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.689353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:58.689353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:58.689353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:58.689353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.689915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:58.689915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:58.689915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:58.689915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:58.689915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:58.689915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:58.689915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:58.689915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:58.689915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.689915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:58.689915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:58.689915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.689915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:58.690377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.690377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:58.690695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:58.691135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.691734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:58.691734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.691734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:58.693309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:58.693614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:58.693727 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:58.693846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:37:58.694277 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:58.694577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.695339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:58.695339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.695339 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:58.695339 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:58.695339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.695339 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:58.695339 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.695339 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:58.696072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:58.696512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:58.696512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:58.696512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.696512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:58.696512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:58.696512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.696512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:58.696512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.696512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:58.696911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.696911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:58.696911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:58.696911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:58.696911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:58.696911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:58.696911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:58.696911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:37:58.696911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:58.696911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:58.696911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.696911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:58.697328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:58.697969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:58.698353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:58.698860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:58.698860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:58.698860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:58.698860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:58.698860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:58.698860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.698860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:58.698860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:58.698860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.698860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:58.698860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.698860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:58.699902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:58.700222 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:58.700337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:58.700465 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:58.701255 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:58.701609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.702279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:58.702279 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.702279 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:58.702279 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:58.702279 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.702279 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:58.702279 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.702279 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:37:58.702997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:37:58.703388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:58.703388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:37:58.703388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.703388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:37:58.703388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:37:58.703388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.703388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:37:58.703388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.703388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:37:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:58.704172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:58.704564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.704564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:58.704564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.704564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:58.704951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:58.705425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:58.705915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.705915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:58.705915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.705915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:37:58.708454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:37:58.708699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:37:58.708806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:37:58.708957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:37:58.709340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:58.709635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:37:58.709770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.710283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:37:58.710283 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.710283 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:37:58.710283 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:37:58.710283 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.710283 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:37:58.710283 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.710283 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:37:58.711089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.711089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:37:58.711089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:58.711089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:37:58.711089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:37:58.711089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:58.711089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:58.711089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.711089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:58.711089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:58.711089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:37:58.711089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.711426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:37:58.711426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:58.711426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:37:58.711426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:58.711426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:37:58.711426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:37:58.711426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:58.711426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:37:58.711426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.711426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:37:58.711426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:37:58.711426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.711426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:37:58.711757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.711757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:37:58.712081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:37:58.712538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:37:58.713019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:37:58.713019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:37:58.713019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:00.700677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:00.701553 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:00.701553 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:00.702786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:00.741164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:00.742364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:00.742364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:00.742364 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:00.742364 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:00.742364 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:00.742364 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:00.742364 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:00.742364 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:00.742364 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:00.743387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:00.743387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:00.743387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:00.743387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:00.743387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:00.743387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:00.743387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:00.743387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:00.743387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:00.743387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:00.743387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:00.743387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:00.743928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:00.743928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:00.743928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:00.743928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:00.743928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:00.743928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:00.743928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:00.743928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:00.743928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:00.743928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:00.743928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:00.743928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:00.743928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:00.744577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:00.745183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:00.745953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:00.745953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:00.745953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:00.745953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:00.745953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:00.745953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:03.685392 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:03.686085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:03.686085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:03.687011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:03.687371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:03.687634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.688491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:03.688491 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.688491 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:03.688491 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:03.688491 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.688491 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:03.688491 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.688491 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:03.689528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.689528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:03.689528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:03.689528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:03.689528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:03.689528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:03.689528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:03.689528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.689528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:03.689528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:03.689528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:03.689528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.690111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:03.690111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:03.690111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:03.690111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:03.690111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:03.690111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:03.690111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:03.690111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:03.690111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.690111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:03.690111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:03.690111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.690111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:03.690627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:03.691262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.692030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:03.692030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:03.692030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.692030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:03.692030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.692030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:03.693880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:03.694428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:03.694596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:03.694761 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:38:03.695328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:03.695632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.696333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:03.696333 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.696333 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:03.696333 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:03.696333 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.696333 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:03.696333 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.696333 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:03.697273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:03.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:03.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:03.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:03.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:03.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:03.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:03.698081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.698081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:03.698081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:03.698081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:03.698081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:03.698081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:03.698081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:03.698081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:38:03.698081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:03.698081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:03.698081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.698081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:03.698414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:03.698778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.698778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:03.698778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.698778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:03.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:03.699552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:03.700028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.700028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:03.700028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:03.700028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.700028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:03.700028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.700028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:03.700373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:03.700730 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:03.700853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:03.700980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:03.701503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:03.701839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.702638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:03.702638 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.702638 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:03.702638 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:03.702638 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.702638 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:03.702638 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.702638 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:03.703499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:03.703971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:03.703971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:03.703971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.703971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:03.703971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:03.703971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.703971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:03.703971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.703971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:03.704415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.704415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:03.704415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:03.704415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:03.704415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:03.704415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:03.704415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:03.704415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:03.704415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:03.704415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.704415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:03.704745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:03.705134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.705134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:03.705134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.705134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:03.705481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:03.705968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:03.706487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.706487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:03.706487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.706487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:03.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:03.709371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:03.709480 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:03.709653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:03.710036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:03.710426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:03.710629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.711419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:03.711419 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.711419 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:03.711419 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:03.711419 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.711419 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:03.711419 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.711419 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:03.712944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.712944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:03.712944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:03.712944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:03.712944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:03.712944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:03.712944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:03.712944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.712944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:03.712944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:03.712944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:03.712944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.713386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:03.713386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:03.713386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:03.713386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:03.713386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:03.713386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:03.713386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:03.713386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:03.713386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.713386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:03.713386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:03.713386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.713386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:03.713939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.713939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:03.714085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:03.714578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:03.715099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:03.715099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:03.715099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:05.701409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:05.702013 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:05.702262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:05.703614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:07.340989 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:07.341826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:07.342349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:07.342349 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:07.342349 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:07.342349 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:07.342349 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:07.342349 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:07.342349 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:07.342349 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:07.343394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 1.640 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:07.343394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:07.343394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:07.343394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:07.343394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:07.343394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:07.343394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:07.343394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:07.343394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:07.343394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:07.343394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:07.343394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:07.343973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:07.343973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:07.343973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:07.343973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:07.343973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:07.343973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:07.343973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:07.343973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:07.343973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:07.343973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:07.343973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:07.343973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:07.343973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:07.344543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:07.345189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:07.345990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:07.345990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:07.345990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:07.345990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:07.345990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:07.345990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:08.686066 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:08.686722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:08.686722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:08.687423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:08.687557 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:08.687659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.688141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:08.688141 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.688141 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:08.688141 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:08.688141 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.688141 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:08.688141 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.688141 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:08.689542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.689542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:08.689542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:08.689542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:08.689542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:08.689542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:08.689542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:08.689542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.689542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:08.689542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:08.689542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:08.689542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.689998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:08.689998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:08.689998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:08.689998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:08.689998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:08.689998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:08.689998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:08.689998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:08.689998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.689998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:08.689998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:08.689998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.689998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:08.690398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.690398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:08.691164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:08.691788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.692325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:08.692325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.692325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:08.694312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:08.694658 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:08.694797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:08.694881 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:38:08.695936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:08.696295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.697125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:08.697125 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.697125 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:08.697125 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:08.697125 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.697125 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:08.697125 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.697125 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:08.697902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:08.698371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:08.698371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:08.698371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.698371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:08.698371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:08.698371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.698371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:08.698371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.698371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:08.698822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.698822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:08.698822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:08.698822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:08.698822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:08.698822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:08.698822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:08.698822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:38:08.698822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:08.698822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:08.698822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.698822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:08.699892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:08.701157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:08.701526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:08.701997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:08.702397 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:08.702397 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:08.702397 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:08.702397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.703095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:08.703095 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.703095 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:08.703095 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:08.703095 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.703095 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:08.703095 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.703095 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:08.703676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:08.704495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:08.705260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:08.705260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:08.705260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:08.705260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:08.705260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.705260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:08.705260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:08.705260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:08.705260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.705260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:08.705260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:08.705260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:08.705260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:08.706014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:08.706014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:08.706014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:08.706014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:08.706014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.706014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:08.706014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:08.706014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.706014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:08.706014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.706014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:08.706014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:08.706562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.707001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:08.709431 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:08.709612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:08.709725 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:08.709880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:08.710186 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:08.710428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:08.710571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.711057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:08.711057 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.711057 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:08.711057 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:08.711057 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.711057 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:08.711057 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.711057 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:08.711968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.711968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:08.711968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:08.711968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:08.711968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:08.711968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:08.711968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:08.711968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.711968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:08.711968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:08.711968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:08.711968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:08.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:08.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:08.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:08.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:08.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:08.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:08.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:08.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:08.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:08.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:08.712758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.712758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:08.713121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:08.713973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:08.714704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:08.714704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:08.714704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:10.702353 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:10.703134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:10.703134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:10.704126 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:11.725833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:11.726715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:11.726978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:11.726978 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:11.726978 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:11.726978 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:11.726978 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:11.726978 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:11.726978 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:11.726978 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:11.728094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 1.024 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:11.728094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:11.728094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:11.728094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:11.728094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:11.728094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:11.728094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:11.728094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:11.728094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:11.728094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:11.728094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:11.728094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:11.728748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:11.728748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:11.728748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:11.728748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:11.728748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:11.728748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:11.728748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:11.728748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:11.728748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:11.728748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:11.728748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:11.728748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:11.728748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:11.729087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:11.729087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:11.729574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:11.730688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:11.731730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:11.731730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:11.731730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:13.686753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:13.687452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:13.687452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:13.688502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:13.688898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:13.689133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.690020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:13.690020 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.690020 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:13.690020 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:13.690020 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.690020 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:13.690020 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.690020 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:13.691545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.691545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:13.691545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:13.691545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:13.691545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:13.691545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:13.691545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:13.691545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.691545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:13.691545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:13.691545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:13.691545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.692112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:13.692112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:13.692112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:13.692112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:13.692112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:13.692112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:13.692112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:13.692112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:13.692112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.692112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:13.692112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:13.692112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.692112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:13.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.692708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:13.693613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:13.694348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.695150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:13.695150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.695150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:13.695640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:13.695954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:13.696122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:13.696339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:38:13.696828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:13.697095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.697828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:13.697828 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.697828 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:13.697828 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:13.697828 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.697828 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:13.697828 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.697828 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:13.698611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:13.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:13.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:13.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:13.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:13.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:13.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:13.699631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.699631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:13.699631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:13.699631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:13.699631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:13.699631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:13.699631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:13.699631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:38:13.699631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:13.699631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:13.699631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.699631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:13.700027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:13.700431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.700431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:13.700431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.700431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:13.700811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:13.701587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:13.702140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.702140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:13.702140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:13.702140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.702140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:13.702140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.702140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:13.702140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:13.702140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:13.702140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:13.702568 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:13.702568 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:13.702858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.703751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:13.703751 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.703751 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:13.703751 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:13.703751 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.703751 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:13.703751 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.703751 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:13.704553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:13.705040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:13.705040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:13.705040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.705040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:13.705040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:13.705040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.705040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:13.705040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.705040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:13.705462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.705462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:13.705462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:13.705462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:13.705462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:13.705462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:13.705462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:13.705462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:13.705462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:13.705462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.705462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:13.705810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:13.706247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.706247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:13.706247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.706247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:13.706535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:13.707133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:13.707668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.707668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:13.707668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.707668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:13.709725 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:13.709943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:13.710064 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:13.710250 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:13.710731 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:13.711095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:13.711420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.712164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:13.712164 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.712164 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:13.712164 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:13.712164 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.712164 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:13.712164 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.712164 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:13.713149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.713149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:13.713149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:13.713149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:13.713149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:13.713149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:13.713149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:13.713149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.713149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:13.713149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:13.713149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:13.713149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.713532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:13.713532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:13.713532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:13.713532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:13.713532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:13.713532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:13.713532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:13.713532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:13.713532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.713532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:13.713532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:13.713532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.713532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:13.713893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.713893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:13.714293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:13.714866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:13.715399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:13.715399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:13.715399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:15.702939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:15.704030 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:15.704030 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:15.704897 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:15.741431 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:15.741814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:15.742680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:15.742680 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:15.742680 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:15.742680 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:15.742680 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:15.742680 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:15.742680 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:15.742680 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:15.743437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:15.743437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:15.743437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:15.743437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:15.743437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:15.743437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:15.743437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:15.743437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:15.743437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:15.743437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:15.743437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:15.743437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:15.744043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:15.744043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:15.744043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:15.744043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:15.744043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:15.744043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:15.744043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:15.744043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:15.744043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:15.744043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:15.744043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:15.744043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:15.744043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:15.744628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:15.745317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:15.746109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:15.746109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:15.746109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:15.746109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:15.746109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:15.746109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:18.687355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:18.688123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:18.688284 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:18.689418 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:18.689745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:18.689973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.690858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:18.690858 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.690858 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:18.690858 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:18.690858 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.690858 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:18.690858 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.690858 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:18.692516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.692516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:18.692516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:18.692516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:18.692516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:18.692516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:18.692516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:18.692516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.692516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:18.692516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:18.692516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:18.692516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.693342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:18.693342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:18.693342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:18.693342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:18.693342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:18.693342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:18.693342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:18.693342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:18.693342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.693342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:18.693342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:18.693342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.693342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:18.694075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.694075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:18.694327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:18.695038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.695769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:18.695769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.695769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:18.695979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:18.696400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:18.696581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:18.696760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:38:18.697198 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:18.697583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.698348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:18.698348 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.698348 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:18.698348 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:18.698348 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.698348 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:18.698348 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.698348 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:18.699359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:18.699895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:18.699895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:18.699895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.699895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:18.699895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:18.699895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.699895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:18.699895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.699895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:18.700226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.700226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:18.700226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:18.700226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:18.700226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:18.700226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:18.700226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:18.700226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:38:18.700226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:18.700226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:18.700226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.700226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:18.700695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:18.701233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:18.701810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:18.702524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:18.703071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:18.703071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:18.703071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:18.703071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.703374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:18.703374 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.703374 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:18.703374 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:18.703374 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.703374 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:18.703374 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.703374 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:18.704066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:18.704646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:18.704646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:18.704646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.704646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:18.704646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:18.704646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.704646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:18.704646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.704646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:18.704993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.704993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:18.704993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:18.704993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:18.704993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:18.704993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:18.704993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:18.704993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:18.704993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:18.704993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.704993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:18.705447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:18.705967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:18.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:18.707584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:18.707584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:18.707584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:18.707584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:18.707584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.707584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:18.707584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:18.707584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.707584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:18.707584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.707584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:18.709956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:18.710182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:18.710310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:18.710450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:18.710820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:18.711142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:18.711391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.712009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:18.712009 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.712009 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:18.712009 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:18.712009 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.712009 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:18.712009 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.712009 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:18.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:18.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:18.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:18.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:18.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:18.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:18.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:18.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:18.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:18.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.713120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:18.713120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:18.713120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:18.713120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:18.713120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:18.713120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:18.713120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:18.713120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:18.713120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.713120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:18.713120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:18.713120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.713120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:18.713514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.713514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:18.713766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:18.714234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:18.714722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:18.714722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:18.714722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:20.703566 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:20.704157 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:20.704438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:20.705786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:20.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:20.739793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:20.740433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:20.740433 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:20.740433 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:20.740433 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:20.740433 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:20.740433 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:20.740433 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:20.740433 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:20.741397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:20.741397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:20.741397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:20.741397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:20.741397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:20.741397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:20.741397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:20.741397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:20.741397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:20.741397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:20.741397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:20.741397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:20.741767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:20.741767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:20.741767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:20.741767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:20.741767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:20.741767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:20.741767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:20.741767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:20.741767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:20.741767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:20.741767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:20.741767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:20.741767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:20.742120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:20.742120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:20.742519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:20.743500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:20.744491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:20.744491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:20.744491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:23.688110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:23.688828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:23.688900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:23.690152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:23.690590 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:23.690905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.691791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:23.691791 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.691791 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:23.691791 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:23.691791 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.691791 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:23.691791 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.691791 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:23.693314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.693314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:23.693314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:23.693314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:23.693314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:23.693314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:23.693314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:23.693314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.693314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:23.693314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:23.693314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:23.693314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.694000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:23.694000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:23.694000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:23.694000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:23.694000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:23.694000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:23.694000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:23.694000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:23.694000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.694000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:23.694000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:23.694000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.694000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:23.694620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.694620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:23.695294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:23.696028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.696853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:23.696853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.696853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:23.696853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:23.696853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:23.697119 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:23.697189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:38:23.697769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:23.698042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.698756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:23.698756 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.698756 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:23.698756 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:23.698756 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.698756 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:23.698756 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.698756 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:23.699477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:23.699953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:23.699953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:23.699953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.699953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:23.699953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:23.699953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.699953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:23.699953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.699953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:23.700287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.700287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:23.700287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:23.700287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:23.700287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:23.700287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:23.700287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:23.700287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:38:23.700287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:23.700287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:23.700287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.700287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:23.700652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:23.701037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.701037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:23.701037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.701037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:23.701346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:23.701815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:23.702331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.702331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:23.702331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:23.702331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.702331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:23.702331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.702331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:23.702331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:23.702604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:23.702604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:23.702697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:23.702993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:23.703198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.703770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:23.703770 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.703770 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:23.703770 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:23.703770 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.703770 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:23.703770 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.703770 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:23.704560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:23.705180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:23.705180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:23.705180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.705180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:23.705180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:23.705180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.705180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:23.705180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.705180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:23.705448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.705448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:23.705448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:23.705448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:23.705448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:23.705448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:23.705448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:23.705448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:23.705448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:23.705448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.705448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:23.705817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:23.706194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.706194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:23.706194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.706194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:23.706660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:23.707307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:23.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:23.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.707832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:23.710427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:23.710791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:23.710919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:23.711072 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:23.711517 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:23.711923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:23.712143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.712824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:23.712824 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.712824 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:23.712824 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:23.712824 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.712824 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:23.712824 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.712824 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:23.713656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.713656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:23.713656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:23.713656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:23.713656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:23.713656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:23.713656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:23.713656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.713656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:23.713656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:23.713656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:23.713656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.714101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:23.714101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:23.714101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:23.714101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:23.714101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:23.714101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:23.714101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:23.714101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:23.714101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.714101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:23.714101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:23.714101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.714101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:23.714537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.714537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:23.714780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:23.715324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:23.715844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:23.715844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:23.715844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:25.704331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:25.705382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:25.705382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:25.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:25.739302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:25.739608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:25.740385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:25.740385 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:25.740385 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:25.740385 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:25.740385 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:25.740385 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:25.740385 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:25.740385 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:25.741649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:25.741649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:25.741649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:25.741649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:25.741649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:25.741649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:25.741649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:25.741649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:25.741649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:25.741649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:25.741649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:25.741649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:25.742038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:25.742038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:25.742038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:25.742038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:25.742038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:25.742038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:25.742038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:25.742038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:25.742038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:25.742038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:25.742038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:25.742038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:25.742038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:25.742427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:25.742427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:25.742900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:25.744195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:25.744696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:25.744696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:25.744696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:28.693582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:28.693582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:28.693582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:28.693582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:28.693582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:28.693582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:28.694666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:28.695988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:28.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:28.697281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:28.698307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:28.698307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:28.698307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:28.698307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.698307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:28.698307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:28.698307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.698307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:28.698307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.698307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:28.698307 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:28.698307 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:28.698938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:28.698938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:38:28.698938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:28.698938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.699131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:28.699131 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.699131 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:28.699131 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:28.699131 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.699131 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:28.699131 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.699131 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:28.699776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:28.700191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:28.700191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:28.700191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.700191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:28.700191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:28.700191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.700191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:28.700191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.700191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:28.700629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.700629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:28.700629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:28.700629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:28.700629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:28.700629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:28.700629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:28.700629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:38:28.700629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:28.700629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:28.700629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.700629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:28.701363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:28.701738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:28.702123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:28.702637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:28.702637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:28.702637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:28.702637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:28.702637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:28.702637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.702637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:28.702637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:28.702637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.702637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:28.702637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.702637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:28.702637 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:28.703021 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:28.703021 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:28.703021 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:28.703420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:28.703625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.704070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:28.704070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.704070 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:28.704070 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:28.704070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.704070 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:28.704070 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.704070 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:28.704821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:28.705235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:28.705235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:28.705235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.705235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:28.705235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:28.705235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.705235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:28.705235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.705235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:28.705719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.705719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:28.705719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:28.705719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:28.705719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:28.705719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:28.705719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:28.705719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:28.705719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:28.705719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.705719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:28.706020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:28.706413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.706413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:28.706413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.706413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:28.706805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:28.707315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:28.707797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.707797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:28.707797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.707797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:28.710808 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:28.711047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:28.711157 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:28.711338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:28.711572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:28.711736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:28.711837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.712239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:28.712239 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.712239 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:28.712239 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:28.712239 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.712239 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:28.712239 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.712239 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:28.713105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.713105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:28.713105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:28.713105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:28.713105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:28.713105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:28.713105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:28.713105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.713105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:28.713105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:28.713105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:28.713105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.722247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:28.722247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:28.722247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:28.722247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:28.722247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:28.722247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:28.722247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:28.722247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:28.722247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.722247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:28.722247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:28.722247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.722247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:28.722592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:28.722992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:28.723640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:28.723640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:28.723640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:28.723640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:28.723640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:28.723640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:30.760681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:30.760681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:30.760681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:30.760681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:30.760681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:30.760681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:30.761647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:30.762306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:30.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:30.763441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:30.764291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:30.764291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:30.764291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:30.764291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:30.764291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:30.764291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:30.764291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:30.764291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:30.764291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:33.689954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:33.690914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:33.690914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:33.691796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:33.692117 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:33.692401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.693364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:33.693364 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.693364 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:33.693364 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:33.693364 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.693364 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:33.693364 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.693364 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:33.695132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.695132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:33.695132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:33.695132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:33.695132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:33.695132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:33.695132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:33.695132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.695132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:33.695132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:33.695132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:33.695132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.695754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:33.695754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:33.695754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:33.695754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:33.695754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:33.695754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:33.695754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:33.695754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:33.695754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.695754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:33.695754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:33.695754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.695754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:33.696369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.696369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:33.696612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:33.697395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.698324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:33.698324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.698324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:33.698324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:33.698324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:33.698324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:33.698324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:38:33.698324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:33.698738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.698921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:33.698921 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.698921 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:33.698921 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:33.698921 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.698921 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:33.698921 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.698921 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:33.699817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:33.700519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:33.700519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:33.700519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.700519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:33.700519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:33.700519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.700519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:33.700519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.700519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:33.701268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.701268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:33.701268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:33.701268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:33.701268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:33.701268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:33.701268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:33.701268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:38:33.701268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:33.701268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:33.701268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.701268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:33.701843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:33.702618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:33.703228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:33.703945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:33.704399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:33.704399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:33.704399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:33.704399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.704934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:33.704934 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.704934 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:33.704934 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:33.704934 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.704934 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:33.704934 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.704934 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:33.705519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:33.705988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:33.705988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:33.705988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.705988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:33.705988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:33.705988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.705988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:33.705988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.705988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:33.706369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.706369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:33.706369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:33.706369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:33.706369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:33.706369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:33.706369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:33.706369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:33.706369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:33.706369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.706369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:33.706700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:33.707192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.707192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:33.707192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.707192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:33.707392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:33.707879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:33.708438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.708438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:33.708438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.708438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:33.711358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:33.711594 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:33.711594 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:33.711911 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:33.711976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:33.712345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:33.712569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.713161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:33.713161 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.713161 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:33.713161 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:33.713161 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.713161 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:33.713161 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.713161 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:33.713993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.713993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:33.713993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:33.713993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:33.713993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:33.713993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:33.713993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:33.713993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.713993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:33.713993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:33.713993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:33.713993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.714783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:33.714783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:33.714783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:33.714783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:33.714783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:33.714783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:33.714783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:33.714783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:33.714783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.714783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:33.714783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:33.714783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.714783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:33.715780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:33.716728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:33.717573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:33.717573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:33.717573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:33.717573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:33.717573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:33.717573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:35.706483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:35.706483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:35.707641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:35.708036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:35.744710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:35.744997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:35.745851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:35.745851 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:35.745851 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:35.745851 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:35.745851 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:35.745851 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:35.745851 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:35.745851 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:35.746997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:35.746997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:35.746997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:35.746997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:35.746997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:35.746997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:35.746997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:35.746997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:35.746997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:35.746997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:35.746997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:35.746997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:35.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:35.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:35.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:35.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:35.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:35.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:35.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:35.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:35.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:35.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:35.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:35.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:35.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:35.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:35.749065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:35.749540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:35.749540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:35.749540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:35.749540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:35.749540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:35.749540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:38.690916 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:38.691849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:38.691849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:38.692646 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:38.693023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:38.693294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.694504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:38.694504 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.694504 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:38.694504 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:38.694504 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.694504 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:38.694504 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.694504 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:38.695750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.695750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:38.695750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:38.695750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:38.695750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:38.695750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:38.695750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:38.695750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.695750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:38.695750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:38.695750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:38.695750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.696646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:38.696646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:38.696646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:38.696646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:38.696646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:38.696646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:38.696646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:38.696646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:38.696646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.696646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:38.696646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:38.696646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.696646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:38.697572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:38.698637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.700311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:38.700311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:38.700311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.700311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:38.700311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.700311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:38.700311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:38.700311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:38.700311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:38.700311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:38:38.700802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:38.700802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.700802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:38.700802 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.700802 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:38.700802 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:38.700802 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.700802 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:38.700802 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.700802 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:38.701198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:38.701890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:38.701890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:38.701890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.701890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:38.701890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:38.701890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.701890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:38.701890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.701890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:38.702329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.702329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:38.702329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:38.702329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:38.702329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:38.702329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:38.702329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:38.702329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:38:38.702329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:38.702329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:38.702329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.702329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:38.702839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:38.703404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:38.704000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:38.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:38.705470 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:38.705470 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:38.705470 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:38.705470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.706391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:38.706391 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.706391 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:38.706391 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:38.706391 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.706391 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:38.706391 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.706391 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:38.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:38.707997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:38.707997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:38.707997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.707997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:38.707997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:38.707997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.707997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:38.707997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.707997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:38.708355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.708355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:38.708355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:38.708355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:38.708355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:38.708355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:38.708355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:38.708355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:38.708355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:38.708355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.708355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:38.708717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:38.709107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.709107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:38.709107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.709107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:38.709824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:38.710420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:38.710949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.710949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:38.710949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.710949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:38.711974 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:38.712625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:38.712730 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:38.712941 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:38.713789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:38.714348 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:38.714643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.716001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:38.716001 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.716001 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:38.716001 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:38.716001 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.716001 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:38.716001 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.716001 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:38.717030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.717030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:38.717030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:38.717030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:38.717030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:38.717030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:38.717030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:38.717030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.717030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:38.717030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:38.717030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:38.717030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.717651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:38.717651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:38.717651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:38.717651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:38.717651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:38.717651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:38.717651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:38.717651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:38.717651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.717651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:38.717651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:38.717651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.717651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:38.718250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.718250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:38.718549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:38.719075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:38.719736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:38.719736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:38.719736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:40.733190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:40.733190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:40.733190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:40.733190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:40.744765 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:40.745220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:40.746139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:40.746139 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:40.746139 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:40.746139 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:40.746139 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:40.746139 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:40.746139 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:40.746139 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:40.747127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:40.747127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:40.747127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:40.747127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:40.747127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:40.747127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:40.747127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:40.747127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:40.747127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:40.747127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:40.747127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:40.747127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:40.748038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:40.748038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:40.748038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:40.748038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:40.748038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:40.748038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:40.748038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:40.748038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:40.748038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:40.748038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:40.748038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:40.748038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:40.748038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:40.748909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:40.749747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:40.750589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:40.750589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:40.750589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:40.750589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:40.750589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:40.750589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:43.691505 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:43.692900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:43.693295 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:43.694467 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:43.696180 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:43.696180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.696494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:43.696494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.696494 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:43.696494 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:43.696494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.696494 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:43.696494 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.696494 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:43.698258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.698258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:43.698258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:43.698258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:43.698258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:43.698258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:43.698258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:43.698258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.698258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:43.698258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:43.698258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:43.698258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.698829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:43.698829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:43.698829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:43.698829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:43.698829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:43.698829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:43.698829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:43.698829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:43.698829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.698829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:43.698829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:43.698829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.698829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:43.699415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.699415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:43.700072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:43.700825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.702949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:43.702949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.702949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:43.702949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:43.702949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:43.702949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:43.702949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:38:43.702949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:43.703412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.703412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:43.703412 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.703412 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:43.703412 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:43.703412 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.703412 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:43.703412 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.703412 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:43.704095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:43.704543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:43.704543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:43.704543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.704543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:43.704543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:43.704543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.704543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:43.704543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.704543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:43.705272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.705272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:43.705272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:43.705272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:43.705272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:43.705272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:43.705272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:43.705272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:38:43.705272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:43.705272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:43.705272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.705272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:43.705604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:43.705966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.705966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:43.705966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.705966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:43.705966 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:43.706557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:43.706975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:43.707469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.707469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:43.707469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:43.707469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.707469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:43.707469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.707469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:43.707469 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:43.707469 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:43.707469 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:43.708092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:43.708453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.709232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:43.709232 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.709232 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:43.709232 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:43.709232 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.709232 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:43.709232 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.709232 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:43.711196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:43.711659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:43.711659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:43.711659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.711659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:43.711659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:43.711659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.711659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:43.711659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.711659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:43.712973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.712973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:43.712973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:43.712973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:43.712973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:43.712973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:43.712973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:43.712973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:43.712973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:43.712973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.712973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:43.713370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:43.713785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.713785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:43.713785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.713785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:43.715028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:43.715652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:43.717183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.717183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:43.717183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.717183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:43.717183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:43.717183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:43.717183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:43.717183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:43.717460 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:43.717460 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:43.717460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.717460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:43.717460 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.717460 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:43.717460 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:43.717460 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.717460 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:43.717460 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.717460 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:43.717848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.717848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:43.717848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:43.717848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:43.717848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:43.717848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:43.717848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:43.717848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.717848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:43.717848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:43.717848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:43.717848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.718190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:43.718190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:43.718190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:43.718190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:43.718190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:43.718190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:43.718190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:43.718190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:43.718190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.718190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:43.718190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:43.718190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.718190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:43.718554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.718554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:43.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:43.719223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:43.719815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:43.719815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:43.719815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:45.707066 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:45.707649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:45.707822 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:45.709083 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:45.744155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:45.744155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:45.744394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:45.744394 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:45.744394 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:45.744394 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:45.744394 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:45.744394 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:45.744394 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:45.744394 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:45.745120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:45.745120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:45.745120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:45.745120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:45.745120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:45.745120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:45.745120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:45.745120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:45.745120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:45.745120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:45.745120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:45.745120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:45.745534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:45.745534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:45.745534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:45.745534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:45.745534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:45.745534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:45.745534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:45.745534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:45.745534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:45.745534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:45.745534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:45.745534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:45.745534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:45.745894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:45.746355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:45.746892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:45.746892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:45.746892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:45.746892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:45.746892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:45.746892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:48.692932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:48.693745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:48.693745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:48.694704 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:48.695018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:48.695344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.696183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:48.696183 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.696183 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:48.696183 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:48.696183 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.696183 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:48.696183 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.696183 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:48.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:48.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:48.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:48.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:48.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:48.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:48.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:48.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:48.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:48.697681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.698732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:48.698732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:48.698732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:48.698732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:48.698732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:48.698732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:48.698732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:48.698732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:48.698732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.698732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:48.698732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:48.698732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.698732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:48.699322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:48.699894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.700463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:48.700463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:48.700463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.700463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:48.700463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.700463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:48.701079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:48.701283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:48.701457 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:48.701636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:38:48.702068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:48.702386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.703052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:48.703052 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.703052 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:48.703052 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:48.703052 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.703052 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:48.703052 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.703052 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:48.703848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:48.704809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:48.705730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:48.705730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:48.705730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:38:48.705730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:48.705730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:48.705730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.705730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:48.705730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:48.705730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:48.705730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.705730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:48.705730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:48.705730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:48.706564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:48.706564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:48.706564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:48.706564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:48.706564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:48.706564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.706564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:48.706564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:48.706564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.706564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:48.706564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.706564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:48.707200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:48.707599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:48.708036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.708036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:48.708036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:48.708036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.708036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:48.708036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.708036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:48.708036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:48.708036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:48.708036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:48.708793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:48.708793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:48.708793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.708793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:48.708793 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.708793 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:48.708793 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:48.708793 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.708793 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:48.708793 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.708793 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:48.709572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:48.710476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:48.710883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:48.710883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:48.710883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:48.710883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:48.710883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.710883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:48.710883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:48.710883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:48.710883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.710883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:48.710883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:48.710883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:48.710883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:48.711201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:48.711201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:48.711201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:48.711201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:48.711201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.711201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:48.711201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:48.711201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.711201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:48.711201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.711201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:48.711555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:48.712113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:48.713267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.713267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:48.713267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.713267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:48.715735 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:48.716035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:48.716139 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:48.716323 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:48.716709 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:48.717044 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:48.717291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.717942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:48.717942 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.717942 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:48.717942 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:48.717942 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.717942 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:48.717942 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.717942 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:48.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:48.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:48.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:48.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:48.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:48.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:48.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:48.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:48.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:48.718775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.719259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:48.719259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:48.719259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:48.719259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:48.719259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:48.719259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:48.719259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:48.719259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:48.719259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.719259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:48.719259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:48.719259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.719259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:48.719574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.719574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:48.719894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:48.720917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:48.722000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:48.722000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:48.722000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:50.707774 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:50.708640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:50.708640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:50.709699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:50.711521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=260 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:38:50.749459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:50.750335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:50.751133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:50.751133 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:50.751133 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:50.751133 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:50.751133 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:50.751133 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:50.751133 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:50.751133 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:50.751974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:50.751974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:50.751974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:50.751974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:50.751974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:50.751974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:50.751974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:50.751974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:50.751974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:50.751974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:50.751974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:50.751974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:50.752363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:50.752363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:50.752363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:50.752363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:50.752363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:50.752363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:50.752363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:50.752363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:50.752363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:50.752363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:50.752363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:50.752363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:50.752363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:50.752719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:50.753195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:50.753782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:50.753782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:50.753782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:50.753782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:50.753782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:50.753782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:53.693887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:53.694820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:53.694820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:53.695912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:53.696290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:53.696563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.697471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:53.697471 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.697471 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:53.697471 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:53.697471 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.697471 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:53.697471 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.697471 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:53.698644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.698644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:53.698644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:53.698644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:53.698644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:53.698644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:53.698644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:53.698644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.698644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:53.698644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:53.698644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:53.698644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.699278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:53.699278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:53.699278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:53.699278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:53.699278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:53.699278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:53.699278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:53.699278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:53.699278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.699278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:53.699278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:53.699278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.699278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:53.699826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:53.700553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.701389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:53.701389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:53.701389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.701389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:53.701389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.701389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:53.701389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:53.701859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:53.701978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:53.702098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:38:53.702434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:53.702686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.703144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:53.703144 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.703144 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:53.703144 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:53.703144 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.703144 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:53.703144 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.703144 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:53.703948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:53.704502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:53.704502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:53.704502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.704502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:53.704502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:53.704502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.704502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:53.704502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.704502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:53.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:53.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:53.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:53.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:53.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:53.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:53.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:38:53.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:53.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:53.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.704830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:53.705230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:53.705633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.705633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:53.705633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.705633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:53.705866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:53.706366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:53.706906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.706906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:53.706906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:53.706906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.706906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:53.706906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.706906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:53.707301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:53.707583 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:53.707687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:53.707812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:53.708134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:53.708469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.708980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:53.708980 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.708980 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:53.708980 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:53.708980 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.708980 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:53.708980 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.708980 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:53.709735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:53.710232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:53.710232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:53.710232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.710232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:53.710232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:53.710232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.710232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:53.710232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.710232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:53.710638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.710638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:53.710638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:53.710638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:53.710638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:53.710638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:53.710638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:53.710638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:53.710638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:53.710638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.710638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:53.711023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:53.711465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.711465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:53.711465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.711465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:53.711835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:53.712374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:53.712939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.712939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:53.712939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.712939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:53.716230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:53.716547 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:53.716676 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:53.716842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:53.717104 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:53.717317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:53.717537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.718026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:53.718026 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.718026 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:53.718026 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:53.718026 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.718026 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:53.718026 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.718026 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:53.718918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.718918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:53.718918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:53.718918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:53.718918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:53.718918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:53.718918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:53.718918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.718918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:53.718918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:53.718918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:53.718918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.719320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:53.719320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:53.719320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:53.719320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:53.719320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:53.719320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:53.719320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:53.719320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:53.719320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.719320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:53.719320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:53.719320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.719320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:53.719705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.719705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:53.719935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:53.720447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:53.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:53.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:53.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:55.708424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:55.709188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:55.709188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:55.709898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:55.743225 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:55.743558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:55.744197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:55.744197 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:55.744197 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:55.744197 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:55.744197 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:55.744197 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:55.744197 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:55.744197 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:55.745142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:55.745142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:55.745142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:55.745142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:55.745142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:55.745142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:55.745142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:55.745142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:55.745142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:55.745142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:55.745142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:55.745142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:55.746060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:55.746060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:55.746060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:55.746060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:55.746060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:55.746060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:55.746060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:55.746060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:55.746060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:55.746060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:55.746060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:55.746060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:55.746060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:55.806093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:55.806803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:55.807682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:55.807682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:55.807682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:55.807682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:55.807682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:55.807682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:58.694889 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:58.695813 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:58.695813 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:58.696384 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:58.696622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:58.696805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.697684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:58.697684 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.697684 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:58.697684 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:58.697684 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.697684 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:58.697684 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.697684 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:58.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:58.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:58.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:58.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:58.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:58.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:58.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:58.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:58.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:58.699042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.699890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:58.699890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:58.699890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:58.699890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:58.699890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:58.699890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:58.699890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:58.699890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:58.699890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.699890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:58.699890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:58.699890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.699890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:58.700558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:58.701331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.702208 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:58.702208 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:58.702208 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.702208 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:58.702208 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.702208 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:58.702208 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:58.702208 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:58.702208 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:58.702712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:38:58.702712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:58.702832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.703251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:58.703251 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.703251 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:58.703251 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:58.703251 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.703251 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:58.703251 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.703251 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:58.703882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:58.704228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:58.704228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:58.704228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.704228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:58.704228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:58.704228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.704228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:58.704228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.704228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:58.704643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.704643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:58.704643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:58.704643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:58.704643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:58.704643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:58.704643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:58.704643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:38:58.704643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:58.704643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:58.704643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.704643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:58.704898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:58.705163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.705163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:58.705163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.705163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:58.705616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:58.705937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:58.706308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.706308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:58.706308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:58.706308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.706308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:58.706308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.706308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:58.707765 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:58.708035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:58.708136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:58.708276 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:58.708544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:58.708808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.709467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:58.709467 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.709467 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:58.709467 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:58.709467 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.709467 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:58.709467 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.709467 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:38:58.710094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:58.720126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:58.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:58.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:58.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:58.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:58.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:58.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:58.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:58.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:58.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:58.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:58.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:58.721386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:58.721386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:58.721386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:58.721386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:58.721386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.721386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:58.721386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:58.721386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.721386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:58.721386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.721386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:58.721386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:38:58.721926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.722765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:38:58.723967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:38:58.723967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:38:58.723967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:38:58.723967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:38:58.723967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:58.723967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:38:58.724393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:58.725115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:38:58.725717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.725717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:38:58.725717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:38:58.725717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.725717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:38:58.725717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.725717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:38:58.725717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.725717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:38:58.725717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:38:58.725717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:38:58.725717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:38:58.725717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:38:58.726272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:38:58.727077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:38:58.727077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:38:58.727077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:38:58.727077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:38:58.727077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:38:58.727077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:38:58.727077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:38:58.727077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:38:58.727077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:38:58.727077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:38:58.727077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:38:58.727077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:00.709100 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:00.710010 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:00.710010 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:00.711110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:00.746228 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:00.746598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:00.747369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:00.747369 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:00.747369 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:00.747369 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:00.747369 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:00.747369 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:00.747369 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:00.747369 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:00.748626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:00.748626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:00.748626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:00.748626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:00.748626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:00.748626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:00.748626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:00.748626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:00.748626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:00.748626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:00.748626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:00.748626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:00.749277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:00.749277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:00.749277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:00.749277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:00.749277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:00.749277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:00.749277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:00.749277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:00.749277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:00.749277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:00.749277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:00.749277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:00.749277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:00.750509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:00.751421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:00.752604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:00.752604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:00.752604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:00.752604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:00.752604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:00.752604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:03.695615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:03.696701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:03.696795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:03.698164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:03.698652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:03.698911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.699747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:03.699747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.699747 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:03.699747 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:03.699747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.699747 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:03.699747 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.699747 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:03.700505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.700505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:03.700505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:03.700505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:03.700505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:03.700505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:03.700505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:03.700505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.700505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:03.700505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:03.700505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:03.700505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:03.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:03.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:03.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:03.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:03.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:03.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:03.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:03.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:03.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:03.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:03.701786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:03.702426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.703300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:03.703300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:03.703300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.703300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:03.703300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.703300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:03.703300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:03.703300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:03.703300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:03.703300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:39:03.703801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:03.703801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.704164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:03.704164 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.704164 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:03.704164 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:03.704164 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.704164 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:03.704164 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.704164 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:03.706008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:03.706913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:03.706913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:03.706913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.706913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:03.706913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:03.706913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.706913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:03.706913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.706913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:03.707981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.707981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:03.707981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:03.707981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:03.707981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:03.707981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:03.707981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:03.707981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:39:03.707981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:03.707981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:03.707981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.707981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:03.708627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:03.709417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:03.710091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:03.711004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:03.711724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:03.711724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:03.711724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:03.711724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.711724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:03.711724 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.711724 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:03.711724 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:03.711724 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.711724 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:03.711724 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.711724 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:03.712518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:03.713402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:03.713914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:03.713914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:03.713914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:03.713914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:03.713914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.713914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:03.713914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:03.713914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:03.713914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.713914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:03.713914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:03.713914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:03.713914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:03.714380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:03.714380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:03.714380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:03.714380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:03.714380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.714380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:03.714380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:03.714380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.714380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:03.714380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.714380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:03.714380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:03.714724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:03.717818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:03.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:03.718133 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:03.718266 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:03.718576 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:03.718820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:03.718990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.719483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:03.719483 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.719483 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:03.719483 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:03.719483 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.719483 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:03.719483 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.719483 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:03.720257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.720257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:03.720257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:03.720257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:03.720257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:03.720257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:03.720257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:03.720257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.720257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:03.720257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:03.720257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:03.720257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.720794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:03.720794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:03.720794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:03.720794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:03.720794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:03.720794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:03.720794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:03.720794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:03.720794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.720794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:03.720794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:03.720794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.720794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:03.721325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:03.721821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:03.722306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:03.722306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:03.722306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:03.722306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:03.722306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:03.722306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:05.709722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:05.710538 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:05.710612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:05.711867 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:05.743425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:05.743767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:05.744613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:05.744613 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:05.744613 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:05.744613 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:05.744613 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:05.744613 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:05.744613 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:05.744613 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:05.745413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:05.745413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:05.745413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:05.745413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:05.745413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:05.745413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:05.745413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:05.745413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:05.745413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:05.745413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:05.745413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:05.745413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:05.745954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:05.745954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:05.745954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:05.745954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:05.745954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:05.745954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:05.745954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:05.745954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:05.745954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:05.745954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:05.745954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:05.745954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:05.745954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:05.746633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:05.747528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:05.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:05.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:05.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:05.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:05.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:05.748673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:08.696958 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:08.698302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:08.698302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:08.699084 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:08.699427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:08.699664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.700676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:08.700676 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.700676 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:08.700676 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:08.700676 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.700676 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:08.700676 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.700676 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:08.702328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.702328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:08.702328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:08.702328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:08.702328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:08.702328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:08.702328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:08.702328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.702328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:08.702328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:08.702328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:08.702328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.702959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:08.702959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:08.702959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:08.702959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:08.702959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:08.702959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:08.702959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:08.702959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:08.702959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.702959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:08.702959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:08.702959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.702959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:08.703640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.703640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:08.704373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:08.705316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.706091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:08.706091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.706091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:08.706091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:08.706091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:08.706091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:08.706091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:39:08.706091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:08.706421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.706855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:08.706855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.706855 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:08.706855 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:08.706855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.706855 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:08.706855 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.706855 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:08.707614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:08.708073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:08.708073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:08.708073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.708073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:08.708073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:08.708073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.708073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:08.708073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.708073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:08.708411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.708411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:08.708411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:08.708411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:08.708411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:08.708411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:08.708411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:08.708411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:39:08.708411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:08.708411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:08.708411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.708411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:08.708781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:08.709202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.709202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:08.709202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.709202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:08.709407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:08.709876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:08.710412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.710412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:08.710412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:08.710412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.710412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:08.710412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.710412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:08.710412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:08.710412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:08.710412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:08.710721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:08.710721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:08.710944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.711604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:08.711604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.711604 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:08.711604 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:08.711604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.711604 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:08.711604 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.711604 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:08.712334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:08.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:08.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:08.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:08.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:08.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:08.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.712790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:08.713155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.713155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:08.713155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:08.713155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:08.713155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:08.713155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:08.713155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:08.713155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:08.713155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:08.713155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.713155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:08.713615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:08.714019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.714019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:08.714019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.714019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:08.714215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:08.714774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:08.715334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.715334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:08.715334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.715334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:08.718336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:08.718588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:08.718695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:08.718840 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:08.719345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:08.719696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:08.719934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.720634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:08.720634 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.720634 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:08.720634 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:08.720634 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.720634 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:08.720634 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.720634 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:08.721495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.721495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:08.721495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:08.721495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:08.721495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:08.721495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:08.721495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:08.721495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.721495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:08.721495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:08.721495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:08.721495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.721896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:08.721896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:08.721896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:08.721896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:08.721896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:08.721896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:08.721896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:08.721896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:08.721896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.721896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:08.721896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:08.721896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.721896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:08.722277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.722277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:08.722856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:08.723351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:08.723889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:08.723889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:08.723889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:10.710501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:10.711124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:10.711270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:10.712520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:10.743942 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:10.744449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:10.745565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:10.745565 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:10.745565 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:10.745565 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:10.745565 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:10.745565 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:10.745565 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:10.745565 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:10.746621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:10.746621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:10.746621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:10.746621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:10.746621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:10.746621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:10.746621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:10.746621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:10.746621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:10.746621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:10.746621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:10.746621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:10.747371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:10.747371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:10.747371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:10.747371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:10.747371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:10.747371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:10.747371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:10.747371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:10.747371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:10.747371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:10.747371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:10.747371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:10.747371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:10.747958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:10.748701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:10.749585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:10.749585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:10.749585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:10.749585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:10.749585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:10.749585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:13.697686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:13.698477 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:13.698477 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:13.699064 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:13.699501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:13.699807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.700666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:13.700666 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.700666 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:13.700666 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:13.700666 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.700666 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:13.700666 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.700666 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:13.701439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.701439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:13.701439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:13.701439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:13.701439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:13.701439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:13.701439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:13.701439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.701439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:13.701439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:13.701439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:13.701439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.702019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:13.702019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:13.702019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:13.702019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:13.702019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:13.702019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:13.702019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:13.702019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:13.702019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.702019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:13.702019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:13.702019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.702019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:13.702404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:13.702825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.703456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:13.703456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:13.703456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.703456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:13.703456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.703456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:13.705077 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:13.705379 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:13.705473 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:13.705555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:39:13.706011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:13.706394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.707083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:13.707083 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.707083 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:13.707083 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:13.707083 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.707083 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:13.707083 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.707083 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:13.707625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:13.708043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:13.708043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:13.708043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.708043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:13.708043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:13.708043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.708043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:13.708043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.708043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:13.708320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.708320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:13.708320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:13.708320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:13.708320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:13.708320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:13.708320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:13.708320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:39:13.708320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:13.708320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:13.708320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.708320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:13.708654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:13.709051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:13.709506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:13.710046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:13.710046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:13.710046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:13.710046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:13.710046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:13.710046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.710046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:13.710046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:13.710046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.710046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:13.710046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.710046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:13.710411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:13.710411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:13.710484 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:13.710587 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:13.710897 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:13.711159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.711651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:13.711651 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.711651 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:13.711651 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:13.711651 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.711651 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:13.711651 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.711651 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:13.712167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:13.712604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:13.712604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:13.712604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.712604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:13.712604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:13.712604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.712604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:13.712604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.712604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:13.712868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.712868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:13.712868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:13.712868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:13.712868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:13.712868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:13.712868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:13.712868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:13.712868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:13.712868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.712868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:13.713177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:13.713612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:13.714011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:13.714536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:13.714536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:13.714536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:13.714536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:13.714536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.714536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:13.714536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:13.714536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.714536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:13.714536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.714536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:13.718750 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:13.719003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:13.719080 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:13.719196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:13.719538 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:13.719794 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:13.719972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.720503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:13.720503 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.720503 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:13.720503 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:13.720503 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.720503 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:13.720503 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.720503 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:13.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:13.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:13.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:13.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:13.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:13.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:13.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:13.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:13.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:13.721082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.721492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:13.721492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:13.721492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:13.721492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:13.721492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:13.721492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:13.721492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:13.721492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:13.721492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.721492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:13.721492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:13.721492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.721492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:13.721838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:13.722272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:13.722759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:13.722759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:13.722759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:13.722759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:13.722759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:13.722759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:15.711334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:15.712161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:15.712161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:15.713304 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:15.750795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:15.751043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:15.751769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:15.751769 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:15.751769 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:15.751769 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:15.751769 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:15.751769 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:15.751769 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:15.751769 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:15.753007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:15.753007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:15.753007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:15.753007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:15.753007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:15.753007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:15.753007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:15.753007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:15.753007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:15.753007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:15.753007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:15.753007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:15.753554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:15.753554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:15.753554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:15.753554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:15.753554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:15.753554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:15.753554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:15.753554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:15.753554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:15.753554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:15.753554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:15.753554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:15.753554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:15.753994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:15.754427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:15.755195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:15.755195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:15.755195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:15.755195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:15.755195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:15.755195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:18.698463 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:18.699589 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:18.699706 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:18.700653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:18.700999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:18.701237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.702760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:18.702760 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.702760 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:18.702760 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:18.702760 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.702760 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:18.702760 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.702760 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:18.704435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.704435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:18.704435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:18.704435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:18.704435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:18.704435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:18.704435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:18.704435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.704435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:18.704435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:18.704435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:18.704435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.704939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:18.704939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:18.704939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:18.704939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:18.704939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:18.704939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:18.704939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:18.704939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:18.704939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.704939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:18.704939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:18.704939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.704939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:18.705560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.705560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:18.706538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:18.707092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.707743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:18.707743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.707743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:18.707743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:18.707743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:18.707743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:18.707743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:39:18.709333 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:18.709611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.710431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:18.710431 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.710431 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:18.710431 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:18.710431 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.710431 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:18.710431 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.710431 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:18.711167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:18.711699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:18.711699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:18.711699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.711699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:18.711699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:18.711699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.711699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:18.711699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.711699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:18.711699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:18.712438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.712438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:18.712438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:18.712438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:18.712438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:18.712438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:18.712438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:18.712438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:39:18.712438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:18.712438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:18.712438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.712438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:18.712847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:18.713245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.713245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:18.713245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.713245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:18.713916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:18.714373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:18.714862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.714862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:18.714862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:18.714862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.714862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:18.714862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.714862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:18.714862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:18.714862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:18.714862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:18.715146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:18.715185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:18.715900 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.715900 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:18.715900 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:18.715900 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.715900 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:18.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.715900 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:18.716811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:18.717270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:18.717270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:18.717270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.717270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:18.717270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:18.717270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.717270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:18.717270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.717270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:18.717717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.717717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:18.717717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:18.717717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:18.717717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:18.717717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:18.717717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:18.717717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:18.717717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:18.717717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.717717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:18.718061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:18.718451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.718451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:18.718451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.718451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:18.719021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:18.719533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:18.720045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.720045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:18.720045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.720045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:18.720045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:18.720045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:18.720045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:18.720045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:18.720326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:18.720326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:18.720459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.720982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:18.720982 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.720982 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:18.720982 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:18.720982 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.720982 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:18.720982 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.720982 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:18.721788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.721788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:18.721788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:18.721788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:18.721788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:18.721788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:18.721788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:18.721788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.721788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:18.721788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:18.721788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:18.721788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.722234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:18.722234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:18.722234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:18.722234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:18.722234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:18.722234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:18.722234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:18.722234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:18.722234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.722234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:18.722234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:18.722234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.722234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:18.722611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.722611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:18.722842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:18.723342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:18.723852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:18.723852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:18.723852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:20.711993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:20.712775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:20.712849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:20.714152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:20.749751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:20.750153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:20.751044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:20.751044 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:20.751044 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:20.751044 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:20.751044 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:20.751044 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:20.751044 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:20.751044 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:20.752011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:20.752011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:20.752011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:20.752011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:20.752011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:20.752011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:20.752011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:20.752011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:20.752011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:20.752011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:20.752011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:20.752011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:20.752634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:20.752634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:20.752634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:20.752634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:20.752634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:20.752634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:20.752634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:20.752634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:20.752634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:20.752634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:20.752634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:20.752634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:20.752634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:20.753178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:20.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:20.754723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:20.754723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:20.754723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:20.754723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:20.754723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:20.754723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:23.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:23.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:23.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:23.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:23.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:23.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:23.718685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:23.719279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:23.719655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:23.720039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:23.720572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:23.720572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:23.720572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:23.720572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.720572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:23.720572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:23.720572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.720572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:23.720572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.720572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:23.720572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:23.720572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:23.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:23.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:39:23.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:23.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:23.721742 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.721742 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:23.721742 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:23.721742 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.721742 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:23.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:23.722485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:23.723524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:23.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:23.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:23.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:39:23.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:23.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:23.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:23.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:23.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:23.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:23.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:23.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:23.725093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:23.725093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:23.725093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:23.725093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:23.725093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:23.725093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.725093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:23.725093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:23.725093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.725093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:23.725093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.725093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:23.725805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:23.726746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:23.727855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.727855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:23.727855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:23.727855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.727855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:23.727855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.727855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:23.727855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:23.727855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:23.727855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:23.728347 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:23.728347 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:23.728347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.728347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:23.728347 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.728347 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:23.728347 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:23.728347 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.728347 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:23.728347 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.728347 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:23.728347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:23.728628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.729001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:23.729001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:23.729001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.729001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:23.729001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.729001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:23.729001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.729001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:23.729001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:23.729001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:23.729001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:23.729001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:23.729001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:23.729608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:23.729608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:23.729608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.729608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:23.729608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:23.729608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:23.729608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.729608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:23.729608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:23.729608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:23.729608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:23.729608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:23.729608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:23.730539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:23.730539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:23.730539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.730539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:23.730539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:23.730539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.730539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:23.730539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.730539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:23.730539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.730539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:23.730539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.731262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.732472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:23.733067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:23.733067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:23.733067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:23.733067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:23.733067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:23.733067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:23.733254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:23.733627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:23.733937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.733937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:23.733937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:23.733937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.733937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:23.733937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.733937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:23.733937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.733937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:23.733937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:23.733937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:23.733937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:23.733937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:23.734743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:23.735918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:23.735918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:23.735918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:23.735918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:23.735918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:23.735918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:23.735918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:23.735918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:23.735918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:23.735918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:23.735918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:23.735918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:25.712694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:25.713498 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:25.713498 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:25.714490 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:25.748176 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:25.748643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:25.749484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:25.749484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:25.749484 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:25.749484 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:25.749484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:25.749484 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:25.749484 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:25.749484 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:25.750192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:25.750192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:25.750192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:25.750192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:25.750192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:25.750192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:25.750192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:25.750192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:25.750192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:25.750192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:25.750192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:25.750192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:25.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:25.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:25.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:25.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:25.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:25.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:25.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:25.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:25.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:25.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:25.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:25.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:25.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:25.751288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:25.752061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:25.754050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:25.754050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:25.754050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:25.754050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:25.754050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:25.754050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:28.700518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:28.701164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:28.701389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:28.702507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:28.702855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:28.703237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.704086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:28.704086 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.704086 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:28.704086 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:28.704086 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.704086 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:28.704086 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.704086 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:28.705573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.705573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:28.705573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:28.705573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:28.705573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:28.705573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:28.705573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:28.705573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.705573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:28.705573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:28.705573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:28.705573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:28.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:28.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:28.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:28.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:28.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:28.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:28.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:28.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:28.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:28.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.706137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:28.706714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.706714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:28.707190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:28.707912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.708754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:28.708754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.708754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:28.708754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:28.708754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:28.708754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:28.708754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:39:28.709381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:28.709771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.712480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:28.712480 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.712480 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:28.712480 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:28.712480 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.712480 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:28.712480 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.712480 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:28.713395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:28.713819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:28.713819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:28.713819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.713819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:28.713819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:28.713819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.713819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:28.713819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.713819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:28.714232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.714232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:28.714232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:28.714232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:28.714232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:28.714232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:28.714232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:28.714232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:39:28.714232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:28.714232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:28.714232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.714232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:28.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:28.715013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.715013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:28.715013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.715013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:28.715341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:28.715786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:28.716428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.716428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:28.716428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:28.716428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.716428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:28.716428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.716428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:28.716428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:28.716428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:28.716428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:28.716746 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:28.716746 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:28.716746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.717196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:28.717196 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.717196 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:28.717196 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:28.717196 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.717196 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:28.717196 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.717196 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:28.717867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:28.718294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:28.718294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:28.718294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.718294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:28.718294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:28.718294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.718294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:28.718294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.718294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:28.718739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.718739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:28.718739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:28.718739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:28.718739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:28.718739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:28.718739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:28.718739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:28.718739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:28.718739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.718739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:28.719072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:28.719471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.719471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:28.719471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.719471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:28.719861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:28.720411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:28.720938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.720938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:28.720938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.720938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:28.720938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:28.720938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:28.720938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:28.720938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:28.721192 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:28.721314 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:28.721589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.721992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:28.721992 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.721992 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:28.721992 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:28.721992 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.721992 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:28.721992 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.721992 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:28.722817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.722817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:28.722817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:28.722817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:28.722817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:28.722817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:28.722817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:28.722817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.722817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:28.722817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:28.722817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:28.722817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.723164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:28.723164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:28.723164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:28.723164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:28.723164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:28.723164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:28.723164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:28.723164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:28.723164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.723164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:28.723164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:28.723164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.723164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:28.723532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.723532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:28.723921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:28.724365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:28.724951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:28.724951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:28.724951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:30.713301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:30.713832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:30.714017 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:30.715235 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:30.756334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:30.756703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:30.757876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:30.757876 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:30.757876 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:30.757876 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:30.757876 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:30.757876 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:30.757876 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:30.757876 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:30.759039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:30.759039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:30.759039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:30.759039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:30.759039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:30.759039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:30.759039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:30.759039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:30.759039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:30.759039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:30.759039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:30.759039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:30.759376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:30.759376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:30.759376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:30.759376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:30.759376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:30.759376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:30.759376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:30.759376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:30.759376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:30.759376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:30.759376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:30.759376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:30.759376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:30.759738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:30.759738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:30.760161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:30.760580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:30.761099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:30.761099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:30.761099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:33.701332 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:33.702278 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:33.702278 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:33.703319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:33.703704 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:33.703963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.705047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:33.705047 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.705047 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:33.705047 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:33.705047 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.705047 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:33.705047 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.705047 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:33.706180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.706180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:33.706180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:33.706180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:33.706180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:33.706180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:33.706180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:33.706180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.706180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:33.706180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:33.706180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:33.706180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.706748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:33.706748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:33.706748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:33.706748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:33.706748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:33.706748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:33.706748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:33.706748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:33.706748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.706748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:33.706748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:33.706748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.706748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:33.707322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:33.707944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.708791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:33.708791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:33.708791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.708791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:33.708791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.708791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:33.708791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:33.708791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:33.708791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:33.708791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:39:33.710534 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:33.710534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.711230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:33.711230 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.711230 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:33.711230 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:33.711230 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.711230 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:33.711230 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.711230 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:33.712099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:33.712781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:33.712781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:33.712781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.712781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:33.712781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:33.712781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.712781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:33.712781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.712781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:33.713382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.713382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:33.713382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:33.713382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:33.713382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:33.713382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:33.713382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:33.713382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:39:33.713382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:33.713382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:33.713382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.713382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:33.713930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:33.714621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:33.715288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:33.716302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:33.716302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:33.716302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:33.716302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:33.716302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:33.716302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.716302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:33.716302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:33.716302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.716302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:33.716302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.716302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:33.716302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:33.716855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:33.716855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:33.716855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:33.717403 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:33.717860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.718506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:33.718506 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.718506 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:33.718506 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:33.718506 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.718506 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:33.718506 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.718506 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:33.719288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:33.719599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:33.719599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:33.719599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.719599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:33.719599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:33.719599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.719599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:33.719599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.719599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:33.720156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.720156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:33.720156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:33.720156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:33.720156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:33.720156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:33.720156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:33.720156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:33.720156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:33.720156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.720156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:33.720397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:33.720660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.720660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:33.720660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.720660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:33.721421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:33.721755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:33.722121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.722121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:33.722121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.722121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:33.722121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:33.722121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:33.722291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:33.722443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:33.722849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:33.723088 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:33.723265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.724022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:33.724022 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.724022 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:33.724022 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:33.724022 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.724022 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:33.724022 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.724022 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:33.725074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.725074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:33.725074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:33.725074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:33.725074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:33.725074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:33.725074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:33.725074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.725074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:33.725074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:33.725074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:33.725074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.725336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:33.725336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:33.725336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:33.725336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:33.725336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:33.725336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:33.725336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:33.725336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:33.725336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.725336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:33.725336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:33.725336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.725336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:33.725583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.725583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:33.726170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:33.726533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:33.726884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:33.726884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:33.726884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:35.714640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:35.714640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:35.716526 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:35.716526 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:35.755358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:35.755358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:35.756433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:35.756433 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:35.756433 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:35.756433 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:35.756433 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:35.756433 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:35.756433 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:35.756433 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:35.757623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:35.757623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:35.757623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:35.757623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:35.757623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:35.757623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:35.757623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:35.757623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:35.757623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:35.757623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:35.757623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:35.757623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:35.758146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:35.758146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:35.758146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:35.758146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:35.758146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:35.758146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:35.758146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:35.758146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:35.758146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:35.758146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:35.758146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:35.758146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:35.758146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:35.758660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:35.758660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:35.758838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:35.759352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:35.760146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:35.760146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:35.760146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:38.701984 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:38.703912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:38.703912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:38.704780 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:38.705144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:38.705416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.706368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:38.706368 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.706368 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:38.706368 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:38.706368 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.706368 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:38.706368 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.706368 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:38.707386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.707386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:38.707386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:38.707386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:38.707386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:38.707386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:38.707386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:38.707386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.707386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:38.707386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:38.707386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:38.707386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.707752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:38.707752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:38.707752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:38.707752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:38.707752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:38.707752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:38.707752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:38.707752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:38.707752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.707752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:38.707752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:38.707752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.707752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:38.708069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.708069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:38.709038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:38.710674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.711437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:38.711437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.711437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:38.711437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:38.711437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:38.711437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:38.711437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:39:38.711437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:38.712222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.712836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:38.713147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.713147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:38.713147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:38.713147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:38.713147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:38.713147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:38.713147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:38.713147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:39:38.713147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:38.713147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:38.713147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.713147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:38.714622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:38.715470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:38.715729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:38.716066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:38.716066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:38.716066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:38.716066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:38.716066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:38.716066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.716066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:38.716066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:38.716066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.716066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:38.716066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.716066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:38.716697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:38.716979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:38.717110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:38.717263 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:38.717564 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:38.717858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.718369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:38.718369 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.718369 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:38.718369 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:38.718369 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.718369 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:38.718369 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.718369 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:38.719087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:38.719665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:38.719665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:38.719665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.719665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:38.719665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:38.719665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.719665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:38.719665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.719665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:38.719990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.719990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:38.719990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:38.719990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:38.719990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:38.719990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:38.719990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:38.719990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:38.719990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:38.719990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.719990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:38.720338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:38.720708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.720708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:38.720708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.720708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:38.721399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:38.721850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:38.722341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.722341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:38.722341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.722341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:38.722341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:38.722713 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:38.722910 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:38.723400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:38.723836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:38.724178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:38.724427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.725326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:38.725326 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.725326 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:38.725326 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:38.725326 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.725326 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:38.725326 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.725326 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:38.726793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.726793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:38.726793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:38.726793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:38.726793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:38.726793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:38.726793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:38.726793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.726793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:38.726793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:38.726793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:38.726793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.727155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:38.727155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:38.727155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:38.727155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:38.727155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:38.727155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:38.727155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:38.727155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:38.727155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.727155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:38.727155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:38.727155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.727155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:38.727611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.727611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:38.728457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:38.728911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:38.729801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:38.729801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:38.729801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:40.714839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:40.715724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:40.715724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:40.717609 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:40.755718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:40.755924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:40.756624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:40.756624 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:40.756624 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:40.756624 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:40.756624 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:40.756624 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:40.756624 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:40.756624 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:40.757355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:40.757355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:40.757355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:40.757355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:40.757355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:40.757355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:40.757355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:40.757355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:40.757355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:40.757355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:40.757355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:40.757355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:40.757792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:40.757792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:40.757792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:40.757792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:40.757792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:40.757792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:40.757792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:40.757792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:40.757792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:40.757792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:40.757792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:40.757792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:40.757792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:40.758285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:40.758689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:40.759249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:40.759249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:40.759249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:40.759249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:40.759249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:40.759249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:43.703014 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:43.703935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:43.704175 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:43.705336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:43.705681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:43.705943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.706914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:43.706914 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.706914 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:43.706914 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:43.706914 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.706914 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:43.706914 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.706914 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:43.708376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.708376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:43.708376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:43.708376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:43.708376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:43.708376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:43.708376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:43.708376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.708376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:43.708376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:43.708376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:43.708376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.712489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:43.712489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:43.712489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:43.712489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:43.712489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:43.712489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:43.712489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:43.712489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:43.712489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.712489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:43.712489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:43.712489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.712489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:43.713535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:43.714429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.715604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:43.715604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:43.715604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.715604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:43.715604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.715604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:43.715604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:43.715604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:43.715604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:43.715604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:43.716265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.717106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:43.717978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.717978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:43.717978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.717978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:43.717978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:43.717978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:43.717978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:43.717978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:43.717978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:43.717978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:39:43.717978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:43.717978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:43.718664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:43.719444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.720282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:43.721249 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:43.722258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:43.722258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:43.722258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:43.722258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:43.722258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.722258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:43.722258 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.722258 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:43.722258 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:43.722616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.723152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:43.723703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:43.723703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:43.723703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:43.723703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:43.723703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:43.723703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:43.723703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:43.723703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.723703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:43.723703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:43.723703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:43.723703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.723703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:43.724197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:43.724197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:43.724197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:43.724197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:43.724197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:43.724197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:43.724197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:43.724197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.724197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:43.724197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:43.724197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.724197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:43.724197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:43.724720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.725367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:43.725732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:43.725732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.725732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:43.725732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.725732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:43.725732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:43.725732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:43.725732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:43.725732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:43.725936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:43.725936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:43.725936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.725936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:43.725936 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.725936 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:43.725936 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:43.725936 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.725936 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:43.725936 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.725936 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:43.725936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:43.726229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:43.726516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:43.726516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:43.726516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:43.726516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:43.726516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:43.726516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.726516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:43.726516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:43.726516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.726516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:43.726516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.726516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:43.726729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:43.727011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:43.727383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:43.727383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:43.727383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:45.715507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:45.716305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:45.716305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:45.717507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:45.750687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:45.750865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:45.751373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:45.751373 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:45.751373 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:45.751373 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:45.751373 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:45.751373 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:45.751373 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:45.751373 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:45.752087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:45.752087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:45.752087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:45.752087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:45.752087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:45.752087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:45.752087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:45.752087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:45.752087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:45.752087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:45.752087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:45.752087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:45.752755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:45.752755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:45.752755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:45.752755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:45.752755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:45.752755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:45.752755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:45.752755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:45.752755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:45.752755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:45.752755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:45.752755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:45.752755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:45.753289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:45.753977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:45.754879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:45.754879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:45.754879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:45.754879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:45.754879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:45.754879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:48.704223 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:48.704963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:48.705424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:48.706651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:48.706985 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:48.707291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.708167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:48.708167 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.708167 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:48.708167 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:48.708167 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.708167 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:48.708167 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.708167 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:48.709731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.709731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:48.709731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:48.709731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:48.709731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:48.709731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:48.709731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:48.709731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.709731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:48.709731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:48.709731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:48.709731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.710197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:48.710197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:48.710197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:48.710197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:48.710197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:48.710197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:48.710197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:48.710197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:48.710197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.710197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:48.710197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:48.710197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.710197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:48.710533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.710533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:48.711433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:48.711869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.712362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:48.712362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.712362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:48.713358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:48.713668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:48.713784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:48.713905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:39:48.714344 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:48.714639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.715343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:48.715343 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.715343 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:48.715343 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:48.715343 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.715343 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:48.715343 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.715343 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:48.716156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:48.716566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:48.716566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:48.716566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.716566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:48.716566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:48.716566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.716566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:48.716566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.716566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:48.717016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.717016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:48.717016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:48.717016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:48.717016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:48.717016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:48.717016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:48.717016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:39:48.717016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:48.717016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:48.717016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.717016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:48.717349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:48.717722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.717722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:48.717722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.717722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:48.718070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:48.718507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:48.718967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.718967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:48.718967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:48.718967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.718967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:48.718967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.718967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:48.718967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:48.718967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:48.718967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:48.719290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:48.719423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:48.719735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.720406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:48.720406 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.720406 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:48.720406 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:48.720406 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.720406 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:48.720406 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.720406 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:48.721141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:48.721646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:48.721646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:48.721646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.721646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:48.721646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:48.721646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.721646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:48.721646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.721646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:48.722111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.722111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:48.722111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:48.722111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:48.722111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:48.722111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:48.722111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:48.722111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:48.722111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:48.722111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.722111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:48.722426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:48.722780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.722780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:48.722780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.722780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:48.723279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:48.723706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:48.724192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.724192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:48.724192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.724192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:48.724192 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:48.724192 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:48.724192 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:48.724192 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:48.724599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:48.724915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:48.725121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.725809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:48.725809 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.725809 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:48.725809 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:48.725809 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.725809 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:48.725809 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.725809 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:48.727566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.727566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:48.727566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:48.727566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:48.727566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:48.727566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:48.727566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:48.727566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.727566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:48.727566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:48.727566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:48.727566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.727919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:48.727919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:48.727919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:48.727919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:48.727919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:48.727919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:48.727919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:48.727919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:48.727919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.727919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:48.727919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:48.727919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.727919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:48.728245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.728245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:48.729465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:48.730106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:48.730582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:48.730582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:48.730582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:50.716343 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:50.717330 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:50.717330 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:50.718276 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:50.750332 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:50.750712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:50.751380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:50.751380 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:50.751380 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:50.751380 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:50.751380 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:50.751380 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:50.751380 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:50.751380 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:50.752344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:50.752344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:50.752344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:50.752344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:50.752344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:50.752344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:50.752344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:50.752344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:50.752344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:50.752344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:50.752344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:50.752344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:50.753849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:50.753849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:50.753849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:50.753849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:50.753849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:50.753849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:50.753849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:50.753849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:50.753849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:50.753849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:50.753849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:50.753849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:50.753849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:50.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:50.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:50.756463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:50.756463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:50.756463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:50.756463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:50.756463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:50.756463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:53.705055 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:53.705668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:53.705668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:53.706684 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:53.707280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:53.707644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.708588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:53.708588 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.708588 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:53.708588 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:53.708588 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.708588 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:53.708588 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.708588 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:53.710194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.710194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:53.710194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:53.710194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:53.710194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:53.710194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:53.710194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:53.710194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.710194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:53.710194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:53.710194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:53.710194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:53.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:53.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:53.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:53.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:53.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:53.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:53.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:53.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:53.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:53.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.710605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:53.711043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.711043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:53.712461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:53.712938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.713491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:53.713491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.713491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:53.713999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:53.714358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:53.714541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:53.714726 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:39:53.715169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:53.715423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.716780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:53.716780 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.716780 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:53.716780 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:53.716780 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.716780 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:53.716780 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.716780 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:53.717557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:53.718033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:53.718033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:53.718033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.718033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:53.718033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:53.718033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.718033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:53.718033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.718033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:53.718386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.718386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:53.718386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:53.718386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:53.718386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:53.718386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:53.718386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:53.718386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:39:53.718386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:53.718386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:53.718386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.718386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:53.718773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:53.719178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.719178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:53.719178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.719178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:53.720077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:53.720565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:53.721057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.721057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:53.721057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:53.721057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.721057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:53.721057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.721057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:53.721057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:53.721057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:53.721057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:53.721376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:53.721376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:53.721480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.722073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:53.722073 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.722073 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:53.722073 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:53.722073 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.722073 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:53.722073 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.722073 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:53.722915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:53.723327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:53.723327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:53.723327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.723327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:53.723327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:53.723327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.723327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:53.723327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.723327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:53.724045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.724045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:53.724045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:53.724045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:53.724045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:53.724045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:53.724045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:53.724045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:53.724045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:53.724045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.724045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:53.724421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:53.724816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.724816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:53.724816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.724816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:53.724816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:53.725268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:53.725763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:53.726295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.726295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:53.726295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.726295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:53.726295 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:53.726295 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:53.726295 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:53.726295 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:53.726562 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:53.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.727161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:53.727161 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.727161 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:53.727161 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:53.727161 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.727161 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:53.727161 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.727161 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:53.727977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.727977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:53.727977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:53.727977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:53.727977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:53.727977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:53.727977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:53.727977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.727977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:53.727977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:53.727977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:53.727977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.728348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:53.728348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:53.728348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:53.728348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:53.728348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:53.728348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:53.728348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:53.728348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:53.728348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.728348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:53.728348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:53.728348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.728348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:53.728723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:53.729167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:53.729734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:53.729734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:53.729734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:53.729734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:53.729734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:53.729734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:55.717265 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:55.718096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:55.718096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:55.718870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:55.753722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:55.753872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:55.754404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:55.754404 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:55.754404 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:55.754404 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:55.754404 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:55.754404 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:55.754404 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:55.754404 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:55.755095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:55.755095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:55.755095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:55.755095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:55.755095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:55.755095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:55.755095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:55.755095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:55.755095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:55.755095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:55.755095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:55.755095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:55.755628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:55.755628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:55.755628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:55.755628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:55.755628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:55.755628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:55.755628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:55.755628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:55.755628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:55.755628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:55.755628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:55.755628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:55.755628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:55.756035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:55.756494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:55.757038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:55.757038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:55.757038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:55.757038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:55.757038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:55.757038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:58.705542 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:58.706016 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:58.706016 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:58.709262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:58.709262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:58.709262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.709262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:58.709262 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.709262 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:58.709262 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:58.709262 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.709262 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:58.709262 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.709262 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:58.709578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.709578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:58.709578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:58.709578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:58.709578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:58.709578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:58.709578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:58.709578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.709578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:58.709578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:58.709578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:58.709578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.709827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:58.709827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:58.709827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:58.709827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:58.709827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:58.709827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:58.709827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:58.709827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:58.709827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.709827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:58.709827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:58.709827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.709827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:58.710070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.710070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:58.710643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:58.710954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.711351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:58.711351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.711351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:58.714381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:58.714662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:58.714844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:58.715005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:39:58.715355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:58.715569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.716128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:58.716128 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.716128 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:58.716128 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:58.716128 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.716128 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:58.716128 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.716128 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:58.719637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:58.720056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:58.720056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:58.720056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.720056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:58.720056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:58.720056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.720056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:58.720056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.720056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:58.720827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.720827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:58.720827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:58.720827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:58.720827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:58.720827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:58.720827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:58.720827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:39:58.720827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:58.720827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:58.720827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.720827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:58.721163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:58.721560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.721560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:58.721560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.721560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:58.722427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:58.722835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:58.723312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.723312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:58.723312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:58.723312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.723312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:58.723312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.723312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:58.723312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:58.723312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:58.723312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:58.723587 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:58.723587 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:58.723979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.724783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:58.724783 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.724783 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:58.724783 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:58.724783 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.724783 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:58.724783 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.724783 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:39:58.725921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:39:58.726311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:58.726311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:39:58.726311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.726311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:39:58.726311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:39:58.726311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.726311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:39:58.726311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.726311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:39:58.726311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:39:58.727185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.727185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:58.727185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:58.727185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:58.727185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:58.727185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:58.727185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:58.727185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:58.727185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:58.727185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.727185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:58.727522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:58.727901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.727901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:58.727901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.727901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:58.728257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:58.728688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:58.729178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.729178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:58.729178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.729178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:39:58.729178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:39:58.729178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:39:58.729178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:39:58.729178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:58.729585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:39:58.729858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.730721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:39:58.730721 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.730721 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:39:58.730721 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:39:58.730721 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.730721 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:39:58.730721 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.730721 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:39:58.731519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.731519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:39:58.731519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:58.731519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:39:58.731519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:39:58.731519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:58.731519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:58.731519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.731519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:58.731519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:58.731519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:39:58.731519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.731842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:39:58.731842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:58.731842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:39:58.731842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:58.731842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:39:58.731842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:39:58.731842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:58.731842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:39:58.731842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.731842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:39:58.731842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:39:58.731842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.731842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:39:58.732201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.732201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:39:58.732488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:39:58.732907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:39:58.733415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:39:58.733415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:39:58.733415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:00.717635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:00.718479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:00.718479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:00.718869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:00.720398 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=280 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:40:00.756668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:00.756875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:00.757478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:00.757478 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:00.757478 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:00.757478 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:00.757478 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:00.757478 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:00.757478 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:00.757478 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:00.758316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:00.758316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:00.758316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:00.758316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:00.758316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:00.758316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:00.758316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:00.758316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:00.758316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:00.758316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:00.758316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:00.758316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:00.759095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:00.759095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:00.759095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:00.759095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:00.759095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:00.759095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:00.759095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:00.759095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:00.759095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:00.759095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:00.759095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:00.759095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:00.759095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:00.759937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:00.760880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:00.762115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:00.762115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:00.762115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:00.762115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:00.762115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:00.762115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:03.706048 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:03.706967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:03.706967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:03.707838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:03.708001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:03.708001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.708893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:03.708893 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.708893 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:03.708893 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:03.708893 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.708893 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:03.708893 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.708893 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:03.709930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.709930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:03.709930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:03.709930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:03.709930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:03.709930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:03.709930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:03.709930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.709930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:03.709930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:03.709930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:03.709930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.710321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:03.710321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:03.710321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:03.710321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:03.710321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:03.710321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:03.710321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:03.710321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:03.710321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.710321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:03.710321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:03.710321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.710321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:03.710679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.710679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:03.712084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:03.712662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.714540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:03.714540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.714540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:03.714905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:03.715277 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:03.715458 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:03.715650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:40:03.717189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:03.717189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.717389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:03.717389 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.717389 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:03.717389 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:03.717389 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.717389 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:03.717389 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.717389 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:03.733747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:03.734537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:03.735470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:03.735470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:03.735470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:40:03.735470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:03.735470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:03.735470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.735470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:03.735470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:03.735470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:03.735470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.735470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:03.735470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:03.735470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:03.736151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:03.736151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:03.736151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:03.736151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:03.736151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:03.736151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.736151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:03.736151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:03.736151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.736151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:03.736151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.736151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:03.736714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:03.743337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:03.745747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.745747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:03.745747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:03.745747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.745747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:03.745747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.745747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:03.745747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:03.745747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:03.745747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:03.747279 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:03.747279 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:03.747279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.747279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:03.747279 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.747279 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:03.747279 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:03.747279 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.747279 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:03.747279 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.747279 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:03.747279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:03.749654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.750683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:03.750683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:03.750683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.750683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:03.750683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.750683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:03.750683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.750683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:03.750683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:03.750683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:03.750683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:03.750683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:03.750683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:03.751294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:03.751294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:03.751294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.751294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:03.751294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:03.751294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:03.751294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.751294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:03.751294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:03.751294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:03.751294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:03.751294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:03.751294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:03.751807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:03.751807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:03.751807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.751807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:03.751807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:03.751807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.751807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:03.751807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.751807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:03.751807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.751807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:03.751807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.752894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.754114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:03.754697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:03.754697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:03.754697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:03.754697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:03.754697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:03.754697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:03.754930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:03.755769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:03.756010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.756010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:03.756010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:03.756010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.756010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:03.756010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.756010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:03.756010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.756010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:03.756010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:03.756010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:03.756010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:03.756010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:03.756279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:03.756633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:03.756633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:03.756633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:03.756633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:03.756633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:03.756633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:03.756633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:03.756633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:03.756633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:03.756633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:03.756633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:03.756633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:05.718337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:05.719308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:05.719308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:05.720485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:05.764538 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:05.764538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:05.764798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:05.764798 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:05.764798 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:05.764798 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:05.764798 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:05.764798 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:05.764798 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:05.764798 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:05.765786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:05.765786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:05.765786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:05.765786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:05.765786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:05.765786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:05.765786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:05.765786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:05.765786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:05.765786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:05.765786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:05.765786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:05.766338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:05.766338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:05.766338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:05.766338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:05.766338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:05.766338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:05.766338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:05.766338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:05.766338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:05.766338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:05.766338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:05.766338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:05.766338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:05.766870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:05.766870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:05.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:05.768361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:05.769114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:05.769114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:05.769114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:08.706688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:08.707546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:08.707546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:08.708857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:08.708857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:08.708857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.708857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:08.708857 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.708857 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:08.708857 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:08.708857 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.708857 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:08.708857 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.708857 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:08.710064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.710064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:08.710064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:08.710064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:08.710064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:08.710064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:08.710064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:08.710064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.710064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:08.710064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:08.710064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:08.710064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.710588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:08.710588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:08.710588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:08.710588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:08.710588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:08.710588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:08.710588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:08.710588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:08.710588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.710588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:08.710588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:08.710588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.710588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:08.710869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.710869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:08.711253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:08.711598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.711970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:08.711970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.711970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:08.715298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:08.715587 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:08.715717 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:08.715842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:40:08.716894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:08.716894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.717009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:08.717009 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.717009 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:08.717009 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:08.717009 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.717009 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:08.717009 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.717009 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:08.717805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:08.718290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:08.718290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:08.718290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.718290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:08.718290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:08.718290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.718290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:08.718290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.718290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:08.718706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.718706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:08.718706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:08.718706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:08.718706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:08.718706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:08.718706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:08.718706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:40:08.718706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:08.718706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:08.718706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.718706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:08.720000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:08.721446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:08.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:08.722885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:08.722885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:08.722885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:08.722885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:08.722885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:08.722885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.722885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:08.722885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:08.722885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.722885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:08.722885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.722885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:08.723174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:08.723500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:08.723500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:08.723649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:08.724740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:08.724740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.724850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:08.724850 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.724850 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:08.724850 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:08.724850 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.724850 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:08.724850 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.724850 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:08.725660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:08.726400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:08.726400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:08.726400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.726400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:08.726400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:08.726400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.726400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:08.726400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.726400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:08.726771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.726771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:08.726771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:08.726771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:08.726771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:08.726771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:08.726771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:08.726771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:08.726771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:08.726771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.726771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:08.727074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:08.727533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:08.728541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:08.729328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:08.729328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:08.729328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:08.729328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:08.729328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.729328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:08.729328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:08.729328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.729328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:08.729328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.729328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:08.729328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:08.729328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:08.729763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:08.729763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:08.730777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:08.730777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:08.730777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.731148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:08.731148 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.731148 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:08.731148 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:08.731148 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.731148 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:08.731148 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.731148 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:08.732728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.732728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:08.732728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:08.732728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:08.732728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:08.732728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:08.732728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:08.732728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.732728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:08.732728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:08.732728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:08.732728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.732992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:08.732992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:08.732992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:08.732992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:08.732992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:08.732992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:08.732992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:08.732992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:08.732992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.732992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:08.732992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:08.732992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.732992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:08.733263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.733263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:08.736831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:08.737182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:08.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:08.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:08.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:10.719185 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:10.720093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:10.720093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:10.720594 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:10.756334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:10.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:10.757924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:10.757924 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:10.757924 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:10.757924 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:10.757924 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:10.757924 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:10.757924 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:10.757924 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:10.758781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:10.758781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:10.758781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:10.758781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:10.758781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:10.758781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:10.758781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:10.758781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:10.758781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:10.758781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:10.758781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:10.758781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:10.759408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:10.759408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:10.759408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:10.759408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:10.759408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:10.759408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:10.759408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:10.759408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:10.759408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:10.759408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:10.759408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:10.759408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:10.759408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:10.759985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:10.760779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:10.761551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:10.761551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:10.761551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:10.761551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:10.761551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:10.761551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:13.707482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:13.708379 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:13.708379 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:13.709578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:13.709963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:13.710079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.710984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:13.710984 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.710984 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:13.710984 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:13.710984 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.710984 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:13.710984 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.710984 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:13.712863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.712863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:13.712863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:13.712863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:13.712863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:13.712863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:13.712863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:13.712863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.712863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:13.712863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:13.712863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:13.712863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.713349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:13.713349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:13.713349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:13.713349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:13.713349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:13.713349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:13.713349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:13.713349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:13.713349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.713349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:13.713349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:13.713349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.713349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:13.713728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.713728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:13.714765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:13.715322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.715821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:13.715821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.715821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:13.715821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:13.716035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:13.716404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:13.716484 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:40:13.716819 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:13.717008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.717695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:13.717695 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.717695 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:13.717695 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:13.717695 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.717695 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:13.717695 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.717695 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:13.718533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:13.718978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:13.718978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:13.718978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.718978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:13.718978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:13.718978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.718978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:13.718978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.718978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:13.719344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.719344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:13.719344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:13.719344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:13.719344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:13.719344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:13.719344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:13.719344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:40:13.719344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:13.719344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:13.719344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.719344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:13.719708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:13.720136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.720136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:13.720136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.720136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:13.720369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:13.720888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:13.721431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.721431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:13.721431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:13.721431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.721431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:13.721431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.721431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:13.723433 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:13.723607 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:13.723723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:13.723977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:13.724332 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:13.724701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.725447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:13.725447 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.725447 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:13.725447 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:13.725447 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.725447 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:13.725447 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.725447 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:13.726291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:13.726690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:13.726690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:13.726690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.726690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:13.726690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:13.726690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.726690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:13.726690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.726690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:13.727183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.727183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:13.727183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:13.727183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:13.727183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:13.727183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:13.727183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:13.727183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:13.727183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:13.727183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.727183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:13.727594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:13.727957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.727957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:13.727957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.727957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:13.728315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:13.728819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:13.729364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.729364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:13.729364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.729364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:13.729364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:13.729364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:13.729364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:13.729364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:13.729828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:13.730219 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:13.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.731313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:13.731313 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.731313 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:13.731313 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:13.731313 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.731313 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:13.731313 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.731313 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:13.732086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.732086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:13.732086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:13.732086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:13.732086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:13.732086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:13.732086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:13.732086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.732086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:13.732086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:13.732086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:13.732086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.732566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:13.732566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:13.732566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:13.732566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:13.732566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:13.732566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:13.732566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:13.732566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:13.732566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.732566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:13.732566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:13.732566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.732566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:13.732893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.732893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:13.733050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:13.733552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:13.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:13.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:13.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:15.719884 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:15.721380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:15.721581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:15.722802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:15.758075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:15.760275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:15.761077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:15.761715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:15.761715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:15.761715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:15.761715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:15.761715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:15.761715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:15.761715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:15.761715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:15.761715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:15.761715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:15.761715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:15.761715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:15.761715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:15.762186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:15.762879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:15.762879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:15.762879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:15.762879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:15.762879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:15.762879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:15.762879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:15.762879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:15.762879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:15.762879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:15.762879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:15.762879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:18.708177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:18.708773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:18.709026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:18.710294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:18.710613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:18.710848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.711719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:18.711719 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.711719 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:18.711719 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:18.711719 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.711719 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:18.711719 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.711719 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:18.713476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.713476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:18.713476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:18.713476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:18.713476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:18.713476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:18.713476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:18.713476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.713476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:18.713476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:18.713476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:18.713476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.713826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:18.713826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:18.713826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:18.713826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:18.713826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:18.713826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:18.713826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:18.713826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:18.713826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.713826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:18.713826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:18.713826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.713826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:18.714166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.714166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:18.715818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:18.716360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.716862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:18.716862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.716862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:18.716862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:18.716862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:18.717145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:18.717437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:40:18.717955 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:18.718277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.718960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:18.718960 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.718960 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:18.718960 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:18.718960 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.718960 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:18.718960 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.718960 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:18.719724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:18.720103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:18.720103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:18.720103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.720103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:18.720103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:18.720103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.720103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:18.720103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.720103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:18.720643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.720643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:18.720643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:18.720643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:18.720643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:18.720643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:18.720643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:18.720643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:40:18.720643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:18.720643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:18.720643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.720643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:18.720983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:18.721367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.721367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:18.721367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.721367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:18.721742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:18.722052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:18.722381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.722381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:18.722381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:18.722381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.722381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:18.722381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.722381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:18.723665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:18.723844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:18.723945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:18.724086 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:18.724557 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:18.724926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.725710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:18.725710 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.725710 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:18.725710 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:18.725710 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.725710 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:18.725710 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.725710 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:18.726426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:18.726909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:18.726909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:18.726909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.726909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:18.726909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:18.726909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.726909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:18.726909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.726909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:18.727268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.727268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:18.727268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:18.727268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:18.727268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:18.727268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:18.727268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:18.727268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:18.727268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:18.727268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.727268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:18.727674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:18.728146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.728146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:18.728146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.728146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:18.728454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:18.729400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:18.729898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.729898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:18.729898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.729898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:18.729898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:18.729898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:18.729898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:18.729898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:18.730156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:18.730458 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:18.730678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.731350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:18.731350 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.731350 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:18.731350 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:18.731350 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.731350 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:18.731350 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.731350 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:18.732133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.732133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:18.732133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:18.732133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:18.732133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:18.732133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:18.732133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:18.732133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.732133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:18.732133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:18.732133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:18.732133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.732568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:18.732568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:18.732568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:18.732568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:18.732568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:18.732568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:18.732568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:18.732568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:18.732568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.732568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:18.732568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:18.732568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.732568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:18.733010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.733010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:18.733154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:18.733692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:18.734250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:18.734250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:18.734250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:20.721984 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:20.726092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:20.726092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:20.726092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:20.794350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:20.794815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.072 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:20.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:20.798496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:20.798821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:20.799160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:20.799664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:20.799664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:20.799664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:20.799664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:20.799664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:20.799664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:20.799664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:20.799664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:20.799664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:23.709595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:23.709595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:23.709595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:23.712950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:23.713259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:23.713541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.715596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:23.715596 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.715596 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:23.715596 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:23.715596 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.715596 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:23.715596 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.715596 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:23.716904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.716904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:23.716904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:23.716904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:23.716904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:23.716904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:23.716904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:23.716904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.716904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:23.716904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:23.716904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:23.716904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.717323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:23.717323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:23.717323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:23.717323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:23.717323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:23.717323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:23.717323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:23.717323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:23.717323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.717323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:23.717323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:23.717323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.717323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:23.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:23.720165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:23.720526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.721195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:23.721195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.721195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:23.721514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:23.721791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:23.722151 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:23.722338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:40:23.722746 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:23.722950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.723767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:23.723767 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.723767 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:23.723767 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:23.723767 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.723767 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:23.723767 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.723767 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:23.724428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:23.724917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:23.724917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:23.724917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.724917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:23.724917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:23.724917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.724917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:23.724917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.724917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:23.725259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:23.726267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.726267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:23.726267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:23.726267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:23.726267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:23.726267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:23.726267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:23.726267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:40:23.726267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:23.726267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:23.726267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.726267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:23.727445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:23.727790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:23.728086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:23.728453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:23.728453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:23.728453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:23.728453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:23.728453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:23.728453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.728453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:23.728453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:23.728453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.728453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:23.728453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.728453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:23.731046 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:23.731227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:23.731400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:23.731656 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:23.732011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:23.732177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.732811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:23.732811 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.732811 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:23.732811 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:23.732811 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.732811 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:23.732811 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.732811 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:23.733405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:23.733736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:23.733736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:23.733736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.733736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:23.733736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:23.733736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.733736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:23.733736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.733736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:23.734818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.734818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:23.734818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:23.734818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:23.734818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:23.734818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:23.734818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:23.734818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:23.734818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:23.734818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.734818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:23.735087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:23.735416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.735416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:23.735416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.735416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:23.736555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:23.737300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:23.737645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.737645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:23.737645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.737645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:23.737645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:23.737645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:23.737645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:23.739845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:23.740138 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:23.740269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.741301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:23.741301 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.741301 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:23.741301 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:23.741301 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.741301 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:23.741301 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.741301 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:23.741559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.741559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:23.741559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:23.741559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:23.741559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:23.741559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:23.741559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:23.741559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.741559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:23.741559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:23.741559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:23.741559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.741863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:23.741863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:23.741863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:23.741863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:23.741863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:23.741863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:23.741863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:23.741863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:23.741863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.741863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:23.741863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:23.741863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.741863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:23.742114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.742114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:23.745147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:23.745543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:23.745930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:23.745930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:23.745930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:25.723824 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:25.724608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:25.724608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:25.725092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:25.762909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:25.763106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:25.763857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:25.763857 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:25.763857 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:25.763857 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:25.763857 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:25.763857 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:25.763857 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:25.763857 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:25.764948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:25.764948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:25.764948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:25.764948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:25.764948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:25.764948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:25.764948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:25.764948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:25.764948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:25.764948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:25.764948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:25.764948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:25.765351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:25.765351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:25.765351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:25.765351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:25.765351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:25.765351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:25.765351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:25.765351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:25.765351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:25.765351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:25.765351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:25.765351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:25.765351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:25.765761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:25.765761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:25.766098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:25.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:25.767345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:25.767345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:25.767345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:28.709503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:28.710079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:28.710079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:28.710667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:28.711013 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:28.711283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.712027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:28.712027 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.712027 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:28.712027 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:28.712027 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.712027 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:28.712027 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.712027 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:28.712960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.712960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:28.712960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:28.712960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:28.712960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:28.712960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:28.712960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:28.712960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.712960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:28.712960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:28.712960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:28.712960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.713605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:28.713605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:28.713605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:28.713605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:28.713605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:28.713605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:28.713605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:28.713605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:28.713605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.713605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:28.713605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:28.713605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.713605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:28.714515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:28.715285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.716040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:28.716040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:28.716040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.716040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:28.716040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.716040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:28.721872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:28.722216 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:28.722298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:28.722465 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:40:28.722865 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:28.723064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.723615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:28.723615 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.723615 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:28.723615 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:28.723615 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.723615 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:28.723615 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.723615 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:28.724540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:28.724933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:28.724933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:28.724933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.724933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:28.724933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:28.724933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.724933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:28.724933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.724933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:28.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:28.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:28.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:28.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:28.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:28.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:28.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:40:28.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:28.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:28.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.725624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:28.725964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:28.726354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.726354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:28.726354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.726354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:28.727840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:28.728585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:28.729191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.729191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:28.729191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:28.729191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.729191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:28.729191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.729191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:28.731191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:28.731672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:28.731672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:28.731981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:28.734653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:28.734996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.735796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:28.735796 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.735796 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:28.735796 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:28.735796 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.735796 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:28.735796 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.735796 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:28.736911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:28.737253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:28.737253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:28.737253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.737253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:28.737253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:28.737253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.737253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:28.737253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.737253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:28.738144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.738144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:28.738144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:28.738144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:28.738144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:28.738144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:28.738144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:28.738144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:28.738144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:28.738144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.738144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:28.738995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:28.739320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.739320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:28.739320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.739320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:28.740326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:28.740779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:28.741259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.741259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:28.741259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.741259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:28.741259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:28.741259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:28.741259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:28.741918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:28.741918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:28.742292 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:28.742292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.742896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:28.742896 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.742896 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:28.742896 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:28.742896 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.742896 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:28.742896 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.742896 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:28.743953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.743953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:28.743953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:28.743953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:28.743953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:28.743953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:28.743953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:28.743953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.743953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:28.743953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:28.743953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:28.743953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.744421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:28.744421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:28.744421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:28.744421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:28.744421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:28.744421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:28.744421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:28.744421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:28.744421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.744421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:28.744421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:28.744421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.744421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:28.744798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.744798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:28.745160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:28.746183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:28.746971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:28.746971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:28.746971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:30.724168 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:30.725082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:30.725418 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:30.727216 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:30.766304 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:30.766538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:30.767113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:30.767113 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:30.767113 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:30.767113 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:30.767113 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:30.767113 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:30.767113 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:30.767113 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:30.767977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:30.767977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:30.767977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:30.767977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:30.767977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:30.767977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:30.767977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:30.767977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:30.767977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:30.767977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:30.767977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:30.767977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:30.768283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:30.768283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:30.768283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:30.768283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:30.768283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:30.768283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:30.768283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:30.768283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:30.768283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:30.768283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:30.768283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:30.768283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:30.768283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:30.768907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:30.769192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:30.769571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:30.769571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:30.769571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:30.769571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:30.769571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:30.769571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:33.709933 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:33.711020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:33.711020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:33.711870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:33.712196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:33.712464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.713376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:33.713376 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.713376 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:33.713376 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:33.713376 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.713376 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:33.713376 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.713376 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:33.714527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.714527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:33.714527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:33.714527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:33.714527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:33.714527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:33.714527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:33.714527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.714527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:33.714527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:33.714527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:33.714527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.714935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:33.714935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:33.714935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:33.714935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:33.714935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:33.714935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:33.714935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:33.714935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:33.714935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.714935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:33.714935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:33.714935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.714935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:33.715178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.715178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:33.715905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:33.716802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.717477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:33.717477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.717477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:33.722425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:33.722779 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:33.722899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:33.723060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:40:33.723922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:33.723922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.724464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:33.724464 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.724464 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:33.724464 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:33.724464 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.724464 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:33.724464 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.724464 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:33.725195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:33.725613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:33.725613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:33.725613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.725613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:33.725613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:33.725613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.725613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:33.725613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.725613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:33.726107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.726107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:33.726107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:33.726107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:33.726107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:33.726107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:33.726107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:33.726107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:40:33.726107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:33.726107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:33.726107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.726107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:33.726462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:33.726802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.726802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:33.726802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.726802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:33.727577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:33.729447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:33.730224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.730224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:33.730224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:33.730224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.730224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:33.730224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.730224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:33.731796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:33.732131 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:33.732268 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:33.732407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:33.732672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:33.732934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.733442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:33.733442 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.733442 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:33.733442 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:33.733442 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.733442 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:33.733442 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.733442 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:33.734133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:33.734447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:33.734447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:33.734447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.734447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:33.734447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:33.734447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.734447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:33.734447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.734447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:33.735637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.735637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:33.735637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:33.735637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:33.735637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:33.735637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:33.735637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:33.735637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:33.735637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:33.735637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.735637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:33.735899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:33.736187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.736187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:33.736187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.736187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:33.736383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:33.736822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:33.737190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.737190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:33.737190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.737190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:33.741008 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:33.741360 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:33.741472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:33.741629 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:33.741854 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:33.742065 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:33.742225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.742653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:33.742653 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.742653 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:33.742653 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:33.742653 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.742653 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:33.742653 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.742653 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:33.743448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.743448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:33.743448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:33.743448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:33.743448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:33.743448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:33.743448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:33.743448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.743448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:33.743448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:33.743448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:33.743448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.743763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:33.743763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:33.743763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:33.743763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:33.743763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:33.743763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:33.743763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:33.743763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:33.743763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.743763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:33.743763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:33.743763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.743763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:33.744068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.744068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:33.745394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:33.745915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:33.746333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:33.746333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:33.746333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:35.725112 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:35.725912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:35.726037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:35.727071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:35.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:35.764194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:35.764930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:35.764930 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:35.764930 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:35.764930 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:35.764930 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:35.764930 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:35.764930 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:35.764930 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:35.765883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:35.765883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:35.765883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:35.765883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:35.765883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:35.765883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:35.765883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:35.765883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:35.765883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:35.765883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:35.765883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:35.765883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:35.766398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:35.766398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:35.766398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:35.766398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:35.766398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:35.766398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:35.766398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:35.766398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:35.766398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:35.766398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:35.766398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:35.766398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:35.766398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:35.767058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:35.767903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:35.768885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:35.768885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:35.768885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:35.768885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:35.768885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:35.768885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:38.710561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:38.711252 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:38.711462 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:38.714829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:38.715608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:38.715852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.716981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:38.716981 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.716981 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:38.716981 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:38.716981 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.716981 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:38.716981 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.716981 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:38.718622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.718622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:38.718622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:38.718622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:38.718622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:38.718622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:38.718622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:38.718622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.718622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:38.718622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:38.718622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:38.718622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.718999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:38.718999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:38.718999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:38.718999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:38.718999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:38.718999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:38.718999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:38.718999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:38.718999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.718999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:38.718999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:38.718999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.718999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:38.719348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.719348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:38.720402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:38.720841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.721367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:38.721367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.721367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:38.722927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:38.723252 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:38.723335 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:38.723437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:40:38.723793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:38.723969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.724294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:38.724294 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.724294 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:38.724294 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:38.724294 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.724294 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:38.724294 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.724294 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:38.724830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:38.725309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:38.725309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:38.725309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.725309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:38.725309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:38.725309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.725309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:38.725309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.725309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:38.725821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.725821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:38.725821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:38.725821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:38.725821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:38.725821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:38.725821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:38.725821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:40:38.725821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:38.725821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:38.725821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.725821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:38.726230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:38.726602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:38.726983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:38.727540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:38.727540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:38.727540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:38.727540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:38.727540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:38.727540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.727540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:38.727540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:38.727540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.727540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:38.727540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.727540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:38.732381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:38.735548 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:38.735665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:38.735817 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:38.736178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:38.736531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.737038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:38.737038 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.737038 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:38.737038 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:38.737038 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.737038 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:38.737038 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.737038 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:38.737867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:38.738368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:38.738368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:38.738368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.738368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:38.738368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:38.738368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.738368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:38.738368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.738368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:38.738863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.738863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:38.738863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:38.738863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:38.738863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:38.738863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:38.738863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:38.738863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:38.738863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:38.738863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.738863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:38.739265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:38.739624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.739624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:38.739624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.739624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:38.740016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:38.740579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:38.741063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.741063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:38.741063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.741063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:38.741517 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:38.742677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:38.742677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:38.742677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:38.742677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:38.742677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:38.742677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.743109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:38.743109 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.743109 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:38.743109 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:38.743109 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.743109 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:38.743109 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.743109 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:38.743696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.743696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:38.743696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:38.743696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:38.743696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:38.743696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:38.743696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:38.743696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.743696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:38.743696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:38.743696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:38.743696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.743977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:38.743977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:38.743977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:38.743977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:38.743977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:38.743977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:38.743977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:38.743977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:38.743977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.743977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:38.743977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:38.743977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.743977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:38.744252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.744252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:38.744386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:38.744708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:38.745070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:38.745070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:38.745070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:40.725830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:40.726418 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:40.726571 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:40.727987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:40.765939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:40.766149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:40.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:40.766787 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:40.766787 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:40.766787 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:40.766787 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:40.766787 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:40.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:40.766787 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:40.768043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:40.768043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:40.768043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:40.768043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:40.768043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:40.768043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:40.768043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:40.768043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:40.768043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:40.768043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:40.768043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:40.768043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:40.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:40.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:40.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:40.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:40.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:40.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:40.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:40.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:40.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:40.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:40.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:40.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:40.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:40.769278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:40.770239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:40.770978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:40.770978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:40.770978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:40.770978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:40.770978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:40.770978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:43.711404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:43.712069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:43.712392 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:43.713107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:43.713412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:43.713904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.714408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:43.714408 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.714408 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:43.714408 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:43.714408 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.714408 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:43.714408 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.714408 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:43.715777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.715777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:43.715777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:43.715777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:43.715777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:43.715777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:43.715777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:43.715777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.715777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:43.715777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:43.715777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:43.715777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.716379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:43.716379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:43.716379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:43.716379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:43.716379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:43.716379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:43.716379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:43.716379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:43.716379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.716379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:43.716379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:43.716379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.716379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:43.716907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.716907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:43.717480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:43.718189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.718990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:43.718990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.718990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:43.723432 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:43.723756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:43.723874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:43.723998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:40:43.724360 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:43.724718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.725847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:43.725847 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.725847 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:43.725847 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:43.725847 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.725847 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:43.725847 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.725847 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:43.726434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:43.726886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:43.726886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:43.726886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.726886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:43.726886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:43.726886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.726886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:43.726886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.726886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:43.727423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.727423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:43.727423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:43.727423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:43.727423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:43.727423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:43.727423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:43.727423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:40:43.727423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:43.727423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:43.727423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.727423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:43.727780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:43.728173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.728173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:43.728173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.728173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:43.728700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:43.729159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:43.729653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.729653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:43.729653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:43.729653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.729653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:43.729653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.729653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:43.735967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:43.736298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:43.736416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:43.736551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:43.736956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:43.737194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.737675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:43.737675 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.737675 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:43.737675 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:43.737675 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.737675 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:43.737675 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.737675 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:43.738402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:43.739029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:43.739029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:43.739029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.739029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:43.739029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:43.739029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.739029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:43.739029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.739029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:43.739468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.739468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:43.739468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:43.739468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:43.739468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:43.739468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:43.739468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:43.739468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:43.739468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:43.739468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.739468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:43.739938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:43.740696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:43.743921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:43.744785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:43.744785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:43.744785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:43.744785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:43.744785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.744785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:43.744785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:43.744785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.744785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:43.744785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.744785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:43.744785 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:43.744785 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:43.745399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:43.745399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:43.745399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:43.745399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:43.745399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.745399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:43.745399 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.745399 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:43.745399 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:43.745817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:43.746457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:43.747054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:43.747746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:43.748469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:43.748469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:43.748469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:43.748469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:43.748469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:43.748469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:45.726351 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:45.727438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:45.727540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:45.728263 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:45.729595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:45.767716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.767875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.768512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:45.768512 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:45.768512 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:45.768512 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:45.768512 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:45.768512 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:45.768512 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.768512 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:45.769600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.769600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:45.769600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:45.769600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:45.769600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:45.769600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:45.769600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:45.769600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.769600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:45.769600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:45.769600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:45.769600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.769915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:45.769915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:45.769915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:45.769915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:45.769915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:45.769915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:45.769915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:45.769915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:45.769915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:45.769915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:45.769915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:45.769915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:45.769915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:45.770172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.770172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:45.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:45.771186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:45.771573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:45.771573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.771573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:45.771573 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:45.771573 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:45.778810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 07 15:40:45.778953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105503) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 07 15:40:45.779078 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:40:45.809815 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, resource={'id': '8b456c85-5e2d-47f8-9886-ebf65932cf32', 'name': 'private', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4a00c537-ed4f-405c-b241-82e7bc2cf3fa', '9fc89727-343d-403a-acd7-db6b4950e7b1'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:25:31Z', 'updated_at': '2026-05-07T15:25:34Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 42799}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.811284 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.811284 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-b0f8152d-626b-440e-9383-919ec905ab38, external_ids_update=None, columns={'tag': 42799}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.811284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.811791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:45.811791 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:45.811791 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:45.811791 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:45.811791 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:45.811791 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:45.811791 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.811791 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:45.812785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.812785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:45.812785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:40:45.812785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:40:45.812785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:40:45.812785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:40:45.812785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:40:45.812785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:45.812785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.812785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:45.812785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:45.813128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:45.813484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:45.813484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.813484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:45.813484 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:40:45.844917 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, resource={'id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'name': 'public', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6aa26064-0a62-428b-a8ed-fef52fc73984', 'adf79775-4dfa-4110-bca0-59be9560299c'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-07T15:25:37Z', 'updated_at': '2026-05-07T15:25:45Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.845189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.845523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-2b8b3cbc-130c-4894-94e1-ab0729ec7077, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.845749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.846526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:45.846526 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:45.846526 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:45.846526 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:45.846526 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:45.846526 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:45.846526 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.846526 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:45.847524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.847524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:45.847524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:40:45.847524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:40:45.847524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:40:45.847524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:40:45.847524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:40:45.847524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:45.847524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.847524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:45.847524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:45.847950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:45.848480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:45.848480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.848480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:45.848480 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:40:45.880586 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, resource={'id': '71aea053-d575-4d13-afc5-e6241ad6a15f', 'name': 'admin_net', 'tenant_id': '74d9c701d5cb4de6a15233057b01e413', 'project_id': '74d9c701d5cb4de6a15233057b01e413', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8e8a2cf7-97a7-4d62-ae4a-fa6c2139e811'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:30:51Z', 'updated_at': '2026-05-07T15:30:52Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 51150}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.880908 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.881334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d5042c49-4c90-465a-b356-c18cf966a58b, external_ids_update=None, columns={'tag': 51150}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.881498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.883072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:45.883072 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:45.883072 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:45.883072 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:45.883072 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:45.883072 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:45.883072 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.883072 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:45.884038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.884038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:45.884038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:40:45.884038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:40:45.884038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:40:45.884038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:40:45.884038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:40:45.884038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:45.884038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.884038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:45.884038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:45.884425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:45.884815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:45.884815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.884815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:45.884815 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:40:45.910839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, resource={'id': '4fb571ad-ed2f-4edf-a199-cb52ff142698', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'tenant_id': '91e1e016e0f8434890282eb48ed58760', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2e6720c7-3159-41b9-80a2-fa6cf6a18808'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:31:26Z', 'updated_at': '2026-05-07T15:31:27Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 29129}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.911101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.911365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13b8a284-1038-44ba-86e4-667f771dfae2, external_ids_update=None, columns={'tag': 29129}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.911365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.911943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:45.911943 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:45.911943 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:45.911943 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:45.911943 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:45.911943 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:45.911943 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.911943 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:45.913277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.913277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:45.913277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:40:45.913277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:40:45.913277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:40:45.913277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:40:45.913277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:40:45.913277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:45.913277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.913277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:45.913277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:45.913732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:45.914147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:45.914147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.914147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:45.914147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:40:45.992935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, resource={'id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'enable_snat': True}, 'gw_port_id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-jseci', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'ip_address': '172.24.5.213', 'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984'}, {'ip_address': '2001:db8::15a', 'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 3, 'tenant_id': '91e1e016e0f8434890282eb48ed58760'}, resource_type=routers, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.993323 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.993614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.993814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:45.993926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.994743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:45.994743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:45.994743 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:45.994743 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:45.994743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:45.994743 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:45.994743 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.994743 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:45.995175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Unable to update router 0938bfb8-1052-49e2-bd69-e5fda45e8a2f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.996727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.996727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:45.996727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:40:45.996727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:40:45.996727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:40:45.996727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:40:45.996727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 07 15:40:45.996727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 07 15:40:45.996727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 07 15:40:45.996727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 07 15:40:45.996727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:45.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:45.998197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:45.998197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:45.998197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:45.998197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:45.998197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:45.998197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:45.998197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:45.998197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:45.998197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:45.998197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:40:46.117239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, resource={'id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'name': '', 'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'project_id': '', 'mac_address': 'fa:16:3e:f6:db:65', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000172501', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:46.117542 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '6aa26064-0a62-428b-a8ed-fef52fc73984 adf79775-4dfa-4110-bca0-59be9560299c', 'neutron:network_name': 'neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.213/24', '2001:db8::15a/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:46.117542 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=b30ffafe-8df3-4064-bcbf-e3d1daad619c, lrouter_port=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:46.117542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:46.117542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:46.117542 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:46.117542 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:46.117542 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:46.117542 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:46.117542 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:46.118242 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:46.118242 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:46.118449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:46.118449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:46.118449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:40:46.118449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:40:46.118449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:40:46.118449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:40:46.118449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 07 15:40:46.118449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:46.118449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:46.118449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:46.118449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:46.118810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:46.119231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:46.119231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:46.119231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:46.119231 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_for_inconsistencies finished in 0.347 seconds
May 07 15:40:48.711886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:48.712660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:48.712660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:48.713928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:48.713928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:48.713928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.714135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:48.714135 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.714135 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:48.714135 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:48.714135 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.714135 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:48.714135 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.714135 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:48.714684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.714684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:48.714684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:48.714684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:48.714684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:48.714684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:48.714684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:48.714684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.714684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:48.714684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:48.714684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:48.714684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.715050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:48.715050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:48.715050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:48.715050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:48.715050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:48.715050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:48.715050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:48.715050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:48.715050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.715050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:48.715050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:48.715050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.715050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:48.715461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:48.715918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.716484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:48.716484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:48.716484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.716484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:48.716484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.716484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:48.723830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:48.724070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:48.724176 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:48.724319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:40:48.725269 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:48.725269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.725376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:48.725376 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.725376 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:48.725376 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:48.725376 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.725376 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:48.725376 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.725376 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:48.726068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:48.726526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:48.726526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:48.726526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.726526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:48.726526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:48.726526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.726526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:48.726526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.726526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:48.726859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.726859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:48.726859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:48.726859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:48.726859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:48.726859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:48.726859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:48.726859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:40:48.726859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:48.726859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:48.726859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.726859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:48.727246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:48.727736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.727736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:48.727736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.727736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:48.727854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:48.728332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:48.728861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.728861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:48.728861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:48.728861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.728861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:48.728861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.728861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:48.736302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:48.736440 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:48.736508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:48.736605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:48.737545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:48.737545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.737659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:48.737659 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.737659 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:48.737659 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:48.737659 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.737659 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:48.737659 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.737659 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:48.738097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:48.738544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:48.738544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:48.738544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.738544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:48.738544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:48.738544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.738544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:48.738544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.738544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:48.738816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.738816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:48.738816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:48.738816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:48.738816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:48.738816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:48.738816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:48.738816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:48.738816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:48.738816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.738816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:48.739167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:48.739607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:48.740003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:48.740544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:48.740544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:48.740544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:48.740544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:48.740544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.740544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:48.740544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:48.740544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.740544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:48.740544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.740544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:48.742121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:48.742279 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:48.742346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:48.742455 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:48.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:48.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:48.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.743651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:48.743651 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.743651 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:48.743651 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:48.743651 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.743651 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:48.743651 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.743651 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:48.744113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.744113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:48.744113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:48.744113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:48.744113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:48.744113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:48.744113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:48.744113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.744113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:48.744113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:48.744113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:48.744113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.744532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:48.744532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:48.744532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:48.744532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:48.744532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:48.744532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:48.744532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:48.744532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:48.744532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.744532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:48.744532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:48.744532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.744532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:48.744990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:48.745433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:48.745962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:48.745962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:48.745962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:48.745962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:48.745962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:48.745962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:50.727699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:50.728344 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:50.728344 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:50.729156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:50.768340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:50.768340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:50.768340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:50.768340 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:50.768340 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:50.768340 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:50.768340 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:50.768340 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:50.768340 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:50.768340 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:50.768738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:50.768738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:50.768738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:50.768738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:50.768738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:50.768738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:50.768738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:50.768738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:50.768738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:50.768738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:50.768738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:50.768738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:50.769091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:50.769091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:50.769091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:50.769091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:50.769091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:50.769091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:50.769091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:50.769091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:50.769091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:50.769091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:50.769091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:50.769091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:50.769091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:50.769449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:50.769838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:50.771020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:50.771020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:50.771020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:50.771020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:50.771020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:50.771020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:53.712753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:53.713567 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:53.713567 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:53.714184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:53.714617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:53.714775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.715743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:53.715743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.715743 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:53.715743 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:53.715743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.715743 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:53.715743 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.715743 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:53.716885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.716885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:53.716885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:53.716885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:53.716885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:53.716885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:53.716885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:53.716885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.716885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:53.716885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:53.716885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:53.716885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.717542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:53.717542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:53.717542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:53.717542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:53.717542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:53.717542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:53.717542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:53.717542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:53.717542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.717542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:53.717542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:53.717542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.717542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:53.718138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:53.718828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.719815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:53.719815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:53.719815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.719815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:53.719815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.719815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:53.724396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:53.724756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:53.724845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:53.724914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:53.728426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.729308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:53.730523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.730523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:53.730523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.730523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:53.730523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:53.730523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:53.730523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:53.730523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:53.730523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:53.730523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:40:53.730523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:53.730523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:53.731201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:53.731861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.732544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.733514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:53.736623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:53.736947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:53.737137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:53.737291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:53.737633 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:53.737892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.738454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:53.738454 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.738454 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:53.738454 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:53.738454 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.738454 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:53.738454 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.738454 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:53.739613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:53.740165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:53.740165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:53.740165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.740165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:53.740165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:53.740165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.740165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:53.740165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.740165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:53.740815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.740815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:53.740815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:53.740815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:53.740815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:53.740815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:53.740815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:53.740815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:53.740815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:53.740815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.740815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:53.741154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:53.741556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.741556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:53.741556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.741556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:53.742410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:53.742990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:53.743474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.743474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:53.743474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.743474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:53.744746 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:53.745169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:53.745345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:53.745559 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:53.746018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:53.746390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:53.746608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.747302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:53.747302 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.747302 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:53.747302 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:53.747302 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.747302 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:53.747302 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.747302 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:53.748688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.748688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:53.748688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:53.748688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:53.748688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:53.748688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:53.748688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:53.748688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.748688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:53.748688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:53.748688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:53.748688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.749078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:53.749078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:53.749078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:53.749078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:53.749078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:53.749078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:53.749078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:53.749078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:53.749078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.749078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:53.749078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:53.749078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.749078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:53.749442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.749442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:53.749829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:53.750277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:53.750718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:53.750718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:53.750718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:55.728366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:55.729101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:55.729101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:55.729844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:55.765726 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:55.766101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:55.767180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:55.767180 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:55.767180 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:55.767180 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:55.767180 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:55.767180 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:55.767180 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:55.767180 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:55.768137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:55.768137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:55.768137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:55.768137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:55.768137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:55.768137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:55.768137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:55.768137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:55.768137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:55.768137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:55.768137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:55.768137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:55.768866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:55.768866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:55.768866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:55.768866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:55.768866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:55.768866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:55.768866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:55.768866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:55.768866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:55.768866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:55.768866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:55.768866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:55.768866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:55.769551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:55.770325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:55.771571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:55.771571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:55.771571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:55.771571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:55.771571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:55.771571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:58.713428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:58.714007 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:58.714232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:58.715696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:58.715696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:58.715696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.715891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:58.715891 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.715891 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:58.715891 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:58.715891 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.715891 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:58.715891 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.715891 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:58.717379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.717379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:58.717379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:58.717379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:58.717379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:58.717379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:58.717379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:58.717379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.717379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:58.717379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:58.717379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:58.717379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:58.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:58.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:58.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:58.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:58.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:58.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:58.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:58.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:58.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:58.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.717682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:58.718007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.718007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:58.719132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:58.719669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.720443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:58.720443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.720443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:58.724938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:58.725176 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:58.725513 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:58.725513 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:40:58.725972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:58.726412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.728648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:58.729579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.730442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:58.730442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:58.730442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:58.730442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:58.730442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:58.730442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:58.730442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:40:58.730442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:58.730442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:58.730442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.730442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:58.730442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:58.730442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.731408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:58.731784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:58.732119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:58.732521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:58.732521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:58.732521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.732521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:58.732521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:58.732521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.732521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:58.732521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.732521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:58.737079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:58.737407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:58.737522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:58.737660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:58.738444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:58.738444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.738607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:58.738607 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.738607 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:58.738607 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:58.738607 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.738607 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:58.738607 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.738607 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:40:58.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:40:58.740502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:58.740502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:40:58.740502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.740502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:40:58.740502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:40:58.740502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.740502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:40:58.740502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.740502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:40:58.740862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.740862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:58.740862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:58.740862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:58.740862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:58.740862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:58.740862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:58.740862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:58.740862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:58.740862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.740862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:58.741123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:58.741424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.741424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:58.741424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.741424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:58.741962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:58.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:58.742843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.742843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:58.742843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.742843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:40:58.745305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:40:58.745945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:40:58.745945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:40:58.745945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:40:58.746346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:58.746668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:40:58.746901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.747468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:40:58.747468 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.747468 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:40:58.747468 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:40:58.747468 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.747468 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:40:58.747468 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.747468 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:40:58.748361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.748361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:40:58.748361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:58.748361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:40:58.748361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:40:58.748361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:58.748361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:58.748361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.748361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:58.748361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:58.748361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:40:58.748361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.748765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:40:58.748765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:58.748765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:40:58.748765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:58.748765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:40:58.748765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:40:58.748765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:58.748765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:40:58.748765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.748765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:40:58.748765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:40:58.748765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.748765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:40:58.749132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.749132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:40:58.750315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:40:58.750959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:40:58.751524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:40:58.751524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:40:58.751524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:00.729074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:00.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:00.729871 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:00.730555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:00.764288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:00.764477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:00.765045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:00.765045 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:00.765045 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:00.765045 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:00.765045 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:00.765045 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:00.765045 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:00.765045 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:00.765823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:00.765823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:00.765823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:00.765823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:00.765823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:00.765823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:00.765823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:00.765823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:00.765823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:00.765823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:00.765823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:00.765823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:00.766444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:00.766444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:00.766444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:00.766444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:00.766444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:00.766444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:00.766444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:00.766444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:00.766444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:00.766444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:00.766444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:00.766444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:00.766444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:00.767022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:00.767763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:00.768573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:00.768573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:00.768573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:00.768573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:00.768573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:00.768573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:03.714064 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:03.714534 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:03.714534 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:03.715258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:03.715497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:03.715677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.718422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:03.718422 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.718422 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:03.718422 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:03.718422 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.718422 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:03.718422 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.718422 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:03.719163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.719163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:03.719163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:03.719163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:03.719163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:03.719163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:03.719163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:03.719163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.719163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:03.719163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:03.719163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:03.719163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.719564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:03.719564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:03.719564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:03.719564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:03.719564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:03.719564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:03.719564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:03.719564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:03.719564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.719564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:03.719564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:03.719564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.719564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:03.719964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:03.720527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.721106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:03.721106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:03.721106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.721106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:03.721106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.721106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:03.725373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:03.725550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:03.725623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:03.725704 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:41:03.726162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:03.726550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.727742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:03.727742 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.727742 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:03.727742 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:03.727742 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.727742 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:03.727742 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.727742 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:03.728427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:03.728789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:03.728789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:03.728789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.728789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:03.728789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:03.728789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.728789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:03.728789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.728789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:03.729051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.729051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:03.729051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:03.729051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:03.729051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:03.729051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:03.729051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:03.729051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:41:03.729051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:03.729051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:03.729051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.729051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:03.729386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:03.729750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.729750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:03.729750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.729750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:03.729885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:03.730294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:03.730732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.730732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:03.730732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:03.730732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.730732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:03.730732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.730732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:03.737580 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:03.737938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:03.738057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:03.738199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:03.738677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:03.739039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.739827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:03.739827 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.739827 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:03.739827 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:03.739827 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.739827 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:03.739827 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.739827 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:03.740888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:03.741339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:03.741339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:03.741339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.741339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:03.741339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:03.741339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.741339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:03.741339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.741339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:03.742259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.742259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:03.742259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:03.742259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:03.742259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:03.742259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:03.742259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:03.742259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:03.742259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:03.742259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.742259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:03.742781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:03.743407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.743407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:03.743407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.743407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:03.745240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:03.746660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:03.747663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.747663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:03.747663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.747663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:03.747663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:03.747663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:03.747663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:03.747663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:03.749326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:03.749326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:03.749326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.750347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:03.750347 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.750347 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:03.750347 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:03.750347 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.750347 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:03.750347 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.750347 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:03.751637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.751637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:03.751637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:03.751637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:03.751637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:03.751637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:03.751637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:03.751637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.751637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:03.751637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:03.751637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:03.751637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:03.752251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.752251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:03.753179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:03.753528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:03.753912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:03.753912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:03.753912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:05.730654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:05.731497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:05.731497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:05.732322 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:05.733966 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=454 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:41:05.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:05.765299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:05.765827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:05.765827 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:05.765827 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:05.765827 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:05.765827 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:05.765827 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:05.765827 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:05.765827 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:05.766583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:05.766583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:05.766583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:05.766583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:05.766583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:05.766583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:05.766583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:05.766583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:05.766583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:05.766583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:05.766583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:05.766583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:05.766947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:05.766947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:05.766947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:05.766947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:05.766947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:05.766947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:05.766947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:05.766947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:05.766947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:05.766947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:05.766947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:05.766947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:05.766947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:05.767312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:05.767743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:05.768299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:05.768299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:05.768299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:05.768299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:05.768299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:05.768299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:08.714646 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:08.716153 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:08.716153 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:08.716977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:08.717295 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:08.717499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.718340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:08.718340 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.718340 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:08.718340 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:08.718340 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.718340 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:08.718340 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.718340 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:08.720175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.720175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:08.720175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:08.720175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:08.720175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:08.720175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:08.720175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:08.720175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.720175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:08.720175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:08.720175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:08.720175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.720486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:08.720486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:08.720486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:08.720486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:08.720486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:08.720486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:08.720486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:08.720486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:08.720486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.720486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:08.720486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:08.720486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.720486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:08.720755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:08.721142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.721573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:08.721573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:08.721573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.721573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:08.721573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.721573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:08.725701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:08.725979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:08.726136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:08.726358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:41:08.727061 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:08.727330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.730926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:08.731602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:08.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:08.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:08.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:08.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:08.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:08.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:41:08.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:08.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:08.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:08.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:08.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.732541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:08.732917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:08.733338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:08.733868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:08.733868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:08.733868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.733868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:08.733868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:08.733868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.733868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:08.733868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.733868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:08.737936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:08.738252 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:08.738467 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:08.738699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:08.739367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:08.739636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.742627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:08.743067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:08.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:08.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:08.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:08.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:08.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:08.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:08.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:08.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:08.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:08.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:08.743527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.744103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:08.744103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:08.744103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:08.744103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:08.744103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:08.744103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:08.744103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:08.744103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:08.744103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.744103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:08.744103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:08.744103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.744103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:08.744599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:08.745046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:08.745590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:08.745590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.745590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:08.745590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:08.745590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.745590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:08.745590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.745590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:08.746100 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:08.746486 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:08.746747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:08.747101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:08.747905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:08.748177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:08.748372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:08.751321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:08.751830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.752189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:08.752189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:08.752189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.752189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:08.752189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.752189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:08.753056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:08.753624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:08.758164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:08.758164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:08.758164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:10.731446 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:10.732773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:10.732865 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:10.733697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:10.767291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:10.767536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:10.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:10.768179 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:10.768179 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:10.768179 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:10.768179 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:10.768179 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:10.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:10.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:10.769157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:10.769157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:10.769157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:10.769157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:10.769157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:10.769157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:10.769157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:10.769157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:10.769157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:10.769157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:10.769157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:10.769157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:10.769425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:10.769425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:10.769425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:10.769425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:10.769425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:10.769425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:10.769425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:10.769425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:10.769425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:10.769425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:10.769425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:10.769425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:10.769425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:10.769658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:10.769658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:10.770219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:10.770510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:10.770843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:10.770843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:10.770843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:13.715542 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:13.716188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:13.716188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:13.716846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:13.717026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:13.717138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.717648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:13.717648 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.717648 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:13.717648 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:13.717648 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.717648 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:13.717648 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.717648 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:13.718639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.718639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:13.718639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:13.718639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:13.718639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:13.718639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:13.718639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:13.718639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.718639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:13.718639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:13.718639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:13.718639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.719640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:13.719640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:13.719640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:13.719640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:13.719640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:13.719640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:13.719640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:13.719640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:13.719640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.719640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:13.719640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:13.719640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.719640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:13.720519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:13.721251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.722079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:13.722079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:13.722079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.722079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:13.722079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.722079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:13.726185 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:13.726540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:13.726681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:13.726817 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:41:13.727058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:13.727199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.727694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:13.727694 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.727694 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:13.727694 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:13.727694 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.727694 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:13.727694 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.727694 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:13.728460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:13.729307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:13.729307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:13.729307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.729307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:13.729307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:13.729307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.729307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:13.729307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.729307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:13.729996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.729996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:13.729996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:13.729996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:13.729996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:13.729996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:13.729996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:13.729996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:41:13.729996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:13.729996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:13.729996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.729996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:13.730589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:13.731226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.731226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:13.731226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.731226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:13.731887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:13.732652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:13.733449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.733449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:13.733449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:13.733449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.733449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:13.733449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.733449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:13.738428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:13.738851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:13.739068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:13.739332 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:13.739641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:13.739788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.740278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:13.740278 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.740278 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:13.740278 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:13.740278 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.740278 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:13.740278 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.740278 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:13.741421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:13.741873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:13.741873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:13.741873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.741873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:13.741873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:13.741873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.741873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:13.741873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.741873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:13.742790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.742790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:13.742790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:13.742790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:13.742790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:13.742790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:13.742790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:13.742790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:13.742790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:13.742790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.742790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:13.743340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:13.743950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.743950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:13.743950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.743950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:13.744523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:13.745277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:13.746019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.746019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:13.746019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.746019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:13.746572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:13.746834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:13.746951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:13.747107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:13.747376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:13.747521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:13.747664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.748137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:13.748137 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.748137 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:13.748137 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:13.748137 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.748137 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:13.748137 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.748137 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:13.749295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.749295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:13.749295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:13.749295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:13.749295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:13.749295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:13.749295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:13.749295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.749295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:13.749295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:13.749295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:13.749295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.749834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:13.749834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:13.749834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:13.749834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:13.749834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:13.749834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:13.749834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:13.749834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:13.749834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.749834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:13.749834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:13.749834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.749834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:13.750385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.750385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:13.750600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:13.751286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:13.752128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:13.752128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:13.752128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:15.732852 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:15.733614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:15.733614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:15.733964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:15.759907 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:15.760092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:15.761318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:15.761318 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:15.761318 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:15.761318 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:15.761318 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:15.761318 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:15.761318 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:15.761318 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:15.762057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:15.762057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:15.762057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:15.762057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:15.762057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:15.762057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:15.762057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:15.762057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:15.762057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:15.762057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:15.762057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:15.762057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:15.762424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:15.762424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:15.762424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:15.762424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:15.762424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:15.762424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:15.762424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:15.762424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:15.762424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:15.762424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:15.762424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:15.762424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:15.762424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:15.762772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:15.763168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:15.763687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:15.763687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:15.763687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:15.763687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:15.763687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:15.763687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:18.716242 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:18.716742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:18.716742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:18.717538 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:18.717783 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:18.717913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.718551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:18.718551 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.718551 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:18.718551 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:18.718551 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.718551 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:18.718551 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.718551 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:18.720150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.720150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:18.720150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:18.720150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:18.720150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:18.720150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:18.720150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:18.720150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.720150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:18.720150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:18.720150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:18.720150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.720515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:18.720515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:18.720515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:18.720515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:18.720515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:18.720515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:18.720515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:18.720515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:18.720515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.720515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:18.720515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:18.720515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.720515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:18.720807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.720807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:18.722158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:18.722534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.722938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:18.722938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.722938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:18.726638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:18.726951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:18.727039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:18.727145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:41:18.727560 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:18.727732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.728323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:18.728323 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.728323 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:18.728323 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:18.728323 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.728323 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:18.728323 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.728323 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:18.728917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:18.729285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:18.729285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:18.729285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.729285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:18.729285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:18.729285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.729285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:18.729285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.729285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:18.729518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.729518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:18.729518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:18.729518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:18.729518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:18.729518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:18.729518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:18.729518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:41:18.729518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:18.729518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:18.729518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.729518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:18.729765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:18.730014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.730014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:18.730014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.730014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:18.730248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:18.730563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:18.730893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.730893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:18.730893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:18.730893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.730893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:18.730893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.730893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:18.739965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:18.740323 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:18.740442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:18.740575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:18.741171 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:18.741527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.742156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:18.742156 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.742156 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:18.742156 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:18.742156 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.742156 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:18.742156 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.742156 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:18.743358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:18.743657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:18.743657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:18.743657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.743657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:18.743657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:18.743657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.743657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:18.743657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.743657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:18.744763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.744763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:18.744763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:18.744763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:18.744763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:18.744763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:18.744763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:18.744763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:18.744763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:18.744763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.744763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:18.745005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:18.745335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.745335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:18.745335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.745335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:18.746525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:18.746890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:18.747259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.747259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:18.747259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.747259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:18.747259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:18.747259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:18.747404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:18.747632 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:18.747968 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:18.748254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:18.748417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.748921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:18.748921 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.748921 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:18.748921 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:18.748921 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.748921 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:18.748921 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.748921 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:18.749784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.749784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:18.749784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:18.749784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:18.749784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:18.749784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:18.749784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:18.749784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.749784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:18.749784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:18.749784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:18.749784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.750046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:18.750046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:18.750046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:18.750046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:18.750046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:18.750046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:18.750046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:18.750046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:18.750046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.750046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:18.750046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:18.750046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.750046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:18.750301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.750301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:18.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:18.751112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:18.751477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:18.751477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:18.751477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:20.733376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:20.733951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:20.733951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:20.734439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:20.771480 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:20.771662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:20.772131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:20.772131 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:20.772131 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:20.772131 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:20.772131 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:20.772131 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:20.772131 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:20.772131 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:20.772762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:20.772762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:20.772762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:20.772762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:20.772762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:20.772762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:20.772762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:20.772762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:20.772762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:20.772762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:20.772762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:20.772762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:20.773160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:20.773160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:20.773160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:20.773160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:20.773160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:20.773160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:20.773160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:20.773160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:20.773160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:20.773160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:20.773160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:20.773160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:20.773160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:20.773438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:20.773746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:20.774097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:20.774097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:20.774097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:20.774097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:20.774097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:20.774097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:23.716679 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:23.718410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:23.718487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:23.719183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:23.719456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:23.719615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.720171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:23.720171 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.720171 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:23.720171 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:23.720171 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.720171 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:23.720171 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.720171 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:23.720986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.720986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:23.720986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:23.720986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:23.720986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:23.720986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:23.720986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:23.720986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.720986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:23.720986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:23.720986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:23.720986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.721406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:23.721406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:23.721406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:23.721406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:23.721406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:23.721406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:23.721406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:23.721406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:23.721406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.721406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:23.721406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:23.721406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.721406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:23.721835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:23.722390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.722942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:23.722942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:23.722942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.722942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:23.722942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.722942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:23.727362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:23.727757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:23.727832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:23.727925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:41:23.728497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:23.728790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.729479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:23.729479 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.729479 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:23.729479 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:23.729479 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.729479 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:23.729479 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.729479 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:23.730179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:23.730516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:23.730516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:23.730516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.730516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:23.730516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:23.730516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.730516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:23.730516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.730516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:23.730792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.730792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:23.730792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:23.730792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:23.730792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:23.730792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:23.730792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:23.730792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:41:23.730792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:23.730792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:23.730792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.730792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:23.731219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:23.731627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:23.732044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:23.734438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:23.734438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:23.734438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:23.734438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:23.734438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:23.734438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.734438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:23.734438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:23.734438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.734438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:23.734438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.734438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:23.740439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:23.740837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:23.740922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:23.741026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:23.741794 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:23.742498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.743245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:23.743245 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.743245 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:23.743245 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:23.743245 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.743245 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:23.743245 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.743245 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:23.743877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:23.744287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:23.744287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:23.744287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.744287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:23.744287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:23.744287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.744287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:23.744287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.744287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:23.744517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.744517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:23.744517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:23.744517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:23.744517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:23.744517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:23.744517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:23.744517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:23.744517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:23.744517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.744517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:23.744810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:23.745149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.745149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:23.745149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.745149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:23.745297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:23.745720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:23.746302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.746302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:23.746302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.746302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:23.747329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:23.748915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:23.748989 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:23.749144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:23.749644 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:23.749972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:23.750388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.751171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:23.751171 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.751171 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:23.751171 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:23.751171 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.751171 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:23.751171 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.751171 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:23.751929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.751929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:23.751929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:23.751929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:23.751929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:23.751929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:23.751929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:23.751929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.751929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:23.751929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:23.751929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:23.751929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.752291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:23.752291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:23.752291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:23.752291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:23.752291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:23.752291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:23.752291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:23.752291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:23.752291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.752291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:23.752291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:23.752291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.752291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:23.752614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.752614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:23.752672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:23.753055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:23.753537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:23.753537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:23.753537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:25.734013 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:25.734606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:25.734711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:25.735614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:25.775582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:25.775824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:25.776349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:25.776349 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:25.776349 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:25.776349 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:25.776349 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:25.776349 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:25.776349 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:25.776349 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:25.777360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:25.777360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:25.777360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:25.777360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:25.777360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:25.777360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:25.777360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:25.777360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:25.777360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:25.777360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:25.777360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:25.777360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:25.777912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:25.777912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:25.777912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:25.777912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:25.777912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:25.777912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:25.777912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:25.777912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:25.777912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:25.777912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:25.777912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:25.777912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:25.777912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:25.778516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:25.779154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:25.779964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:25.779964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:25.779964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:25.779964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:25.779964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:25.779964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:28.718791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:28.719410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:28.719410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:28.720294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:28.720509 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:28.720667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.721290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:28.721290 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.721290 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:28.721290 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:28.721290 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.721290 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:28.721290 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.721290 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:28.723086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.723086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:28.723086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:28.723086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:28.723086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:28.723086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:28.723086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:28.723086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.723086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:28.723086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:28.723086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:28.723086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.723395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:28.723395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:28.723395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:28.723395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:28.723395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:28.723395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:28.723395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:28.723395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:28.723395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.723395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:28.723395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:28.723395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.723395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:28.723653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.723653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:28.724193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:28.724544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.724921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:28.724921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.724921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:28.727809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:28.728049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:28.728172 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:28.728311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:41:28.728642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:28.728827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.729324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:28.729324 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.729324 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:28.729324 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:28.729324 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.729324 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:28.729324 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.729324 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:28.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:28.730676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:28.730676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:28.730676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.730676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:28.730676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:28.730676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.730676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:28.730676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.730676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:28.731079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.731079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:28.731079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:28.731079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:28.731079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:28.731079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:28.731079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:28.731079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:41:28.731079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:28.731079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:28.731079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.731079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:28.732183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:28.732469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:28.732756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:28.733145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:28.733145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:28.733145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:28.733145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:28.733145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:28.733145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.733145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:28.733145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:28.733145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.733145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:28.733145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.733145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:28.740919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:28.741263 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:28.741368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:28.741507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:28.741788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:28.741947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.742375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:28.742375 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.742375 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:28.742375 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:28.742375 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.742375 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:28.742375 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.742375 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:28.743101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:28.743626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:28.743626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:28.743626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.743626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:28.743626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:28.743626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.743626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:28.743626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.743626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:28.744092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.744092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:28.744092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:28.744092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:28.744092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:28.744092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:28.744092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:28.744092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:28.744092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:28.744092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.744092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:28.744374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:28.744647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.744647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:28.744647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.744647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:28.745080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:28.745462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:28.745819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.745819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:28.745819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.745819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:28.748967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:28.749346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:28.749509 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:28.749734 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:28.750087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:28.750320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:28.750462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.751030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:28.751030 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.751030 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:28.751030 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:28.751030 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.751030 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:28.751030 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.751030 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:28.752527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.752527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:28.752527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:28.752527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:28.752527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:28.752527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:28.752527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:28.752527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.752527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:28.752527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:28.752527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:28.752527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.752821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:28.752821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:28.752821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:28.752821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:28.752821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:28.752821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:28.752821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:28.752821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:28.752821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.752821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:28.752821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:28.752821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.752821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:28.753049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.753049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:28.754118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:28.754455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:28.754809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:28.754809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:28.754809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:30.734611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:30.736148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:30.736301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:30.737097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:30.772700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:30.772922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:30.773605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:30.773605 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:30.773605 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:30.773605 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:30.773605 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:30.773605 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:30.773605 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:30.773605 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:30.774340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:30.774340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:30.774340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:30.774340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:30.774340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:30.774340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:30.774340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:30.774340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:30.774340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:30.774340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:30.774340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:30.774340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:30.774900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:30.774900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:30.774900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:30.774900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:30.774900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:30.774900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:30.774900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:30.774900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:30.774900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:30.774900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:30.774900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:30.774900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:30.774900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:30.775485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:30.776143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:30.777247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:30.777247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:30.777247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:30.777247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:30.777247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:30.777247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:33.719806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:33.721747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:33.721747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:33.722022 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:33.722224 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:33.722341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.723970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:33.723970 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.723970 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:33.723970 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:33.723970 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.723970 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:33.723970 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.723970 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:33.724766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.724766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:33.724766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:33.724766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:33.724766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:33.724766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:33.724766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:33.724766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.724766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:33.724766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:33.724766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:33.724766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.726504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:33.726504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:33.726504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:33.726504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:33.726504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:33.726504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:33.726504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:33.726504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:33.726504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.726504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:33.726504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:33.726504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.726504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:33.726918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.726918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:33.728021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:33.729866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.732319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:33.732319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.732319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:33.732319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:33.732319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:33.732319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:33.732319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:41:33.732319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:33.732537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.732930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:33.735224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.735224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:33.735224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:33.735224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:33.735224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:33.735224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:33.735224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:33.735224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:41:33.735224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:33.735224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:33.735224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.735224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:33.736050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:33.736853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:33.737549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:33.738054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:33.738054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:33.738054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:33.738054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:33.738054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:33.738054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.738054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:33.738054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:33.738054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.738054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:33.738054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.738054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:33.741500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:33.741987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:33.741987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:33.741987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:33.742366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:33.742785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.744597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.745015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:33.745696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.745696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:33.745696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:33.745696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:33.745696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:33.745696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:33.745696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:33.745696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:33.745696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:33.745696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.745696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:33.745991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:33.747948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:33.748736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:33.749139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:33.749139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:33.749139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:33.749139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:33.749139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.749139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:33.749139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:33.749139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.749139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:33.749139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.749139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:33.749436 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:33.750252 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:33.750252 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:33.750252 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:33.750488 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:33.750701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:33.750870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:33.754232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:33.754818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.755267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:33.755267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:33.755267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.755267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:33.755267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.755267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:33.755841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:33.756329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:33.756872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:33.756872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:33.756872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:35.736485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:35.739345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:35.739345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:35.739345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:35.769517 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:35.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:35.772487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:35.772852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:35.773742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:35.774232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:35.774828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:35.774828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:35.774828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:35.774828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:35.774828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:35.774828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:35.774828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:35.774828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:35.774828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:38.721639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:38.722634 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:38.722740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:38.723904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:38.724302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:38.724302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.725052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:38.725052 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.725052 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:38.725052 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:38.725052 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.725052 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:38.725052 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.725052 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:38.726756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.726756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:38.726756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:38.726756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:38.726756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:38.726756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:38.726756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:38.726756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.726756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:38.726756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:38.726756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:38.726756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.727188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:38.727188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:38.727188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:38.727188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:38.727188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:38.727188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:38.727188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:38.727188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:38.727188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.727188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:38.727188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:38.727188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.727188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:38.727563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.727563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:38.728682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:38.729166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.729718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:38.729718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.729718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:38.729718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:38.729718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:38.729718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:38.729912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:41:38.730438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:38.730438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.731006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:38.731006 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.731006 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:38.731006 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:38.731006 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.731006 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:38.731006 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.731006 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:38.731729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:38.732195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:38.732195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:38.732195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.732195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:38.732195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:38.732195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.732195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:38.732195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.732195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:38.732485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.732485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:38.732485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:38.732485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:38.732485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:38.732485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:38.732485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:38.732485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:41:38.732485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:38.732485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:38.732485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.732485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:38.732828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:38.733233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:38.733630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:38.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:38.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:38.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:38.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:38.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:38.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:38.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:38.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:38.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:38.742143 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:38.744240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:38.744240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:38.744240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:38.744240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:38.744240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.744240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:38.744240 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.744240 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:38.744240 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:38.744586 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.744586 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:38.744586 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.744586 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:38.745244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:38.745719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:38.745719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:38.745719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.745719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:38.745719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:38.745719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.745719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:38.745719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.745719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:38.746228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.746228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:38.746228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:38.746228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:38.746228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:38.746228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:38.746228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:38.746228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:38.746228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:38.746228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.746228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:38.746611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:38.746975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.746975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:38.746975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.746975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:38.747436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:38.747959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:38.748506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.748506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:38.748506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.748506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:38.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:38.750638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:38.750638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:38.750789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:38.751110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:38.751281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:38.751387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.751839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:38.751839 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.751839 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:38.751839 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:38.751839 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.751839 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:38.751839 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.751839 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:38.753296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.753296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:38.753296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:38.753296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:38.753296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:38.753296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:38.753296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:38.753296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.753296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:38.753296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:38.753296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:38.753296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.753758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:38.753758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:38.753758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:38.753758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:38.753758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:38.753758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:38.753758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:38.753758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:38.753758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.753758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:38.753758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:38.753758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.753758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:38.754177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:38.754654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:38.755104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:38.755104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:38.755104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:38.755104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:38.755104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:38.755104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:40.737643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:40.738453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:40.738453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:40.739939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:40.780444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:40.780708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:40.781407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:40.781407 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:40.781407 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:40.781407 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:40.781407 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:40.781407 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:40.781407 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:40.781407 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:40.782283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:40.782283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:40.782283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:40.782283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:40.782283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:40.782283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:40.782283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:40.782283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:40.782283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:40.782283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:40.782283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:40.782283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:40.782703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:40.782703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:40.782703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:40.782703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:40.782703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:40.782703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:40.782703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:40.782703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:40.782703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:40.782703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:40.782703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:40.782703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:40.782703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:40.783077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:40.783077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:40.783354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:40.783717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:40.784049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:40.784049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:40.784049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:43.722340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:43.723383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:43.723383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:43.723969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:43.724267 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:43.724434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.725186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:43.725186 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.725186 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:43.725186 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:43.725186 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.725186 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:43.725186 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.725186 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:43.726461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.726461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:43.726461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:43.726461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:43.726461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:43.726461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:43.726461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:43.726461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.726461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:43.726461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:43.726461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:43.726461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.726982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:43.726982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:43.726982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:43.726982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:43.726982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:43.726982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:43.726982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:43.726982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:43.726982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.726982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:43.726982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:43.726982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.726982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:43.727338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.727338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:43.728108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:43.728893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.729645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:43.729645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.729645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:43.729645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:43.729821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:43.730166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:43.730166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:41:43.731694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:43.731933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.732534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:43.732534 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.732534 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:43.732534 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:43.732534 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.732534 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:43.732534 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.732534 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:43.733379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:43.733942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:43.733942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:43.733942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.733942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:43.733942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:43.733942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.733942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:43.733942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.733942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:43.734311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.734311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:43.734311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:43.734311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:43.734311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:43.734311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:43.734311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:43.734311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:41:43.734311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:43.734311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:43.734311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.734311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:43.734798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:43.735344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.735344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:43.735344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.735344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:43.736369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:43.736893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:43.737362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.737362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:43.737362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:43.737362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.737362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:43.737362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.737362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:43.744763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:43.744763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:43.744877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:43.744996 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:43.745508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:43.745764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.746495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:43.746495 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.746495 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:43.746495 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:43.746495 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.746495 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:43.746495 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.746495 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:43.747613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:43.748381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:43.748381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:43.748381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.748381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:43.748381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:43.748381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.748381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:43.748381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.748381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:43.749011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.749011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:43.749011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:43.749011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:43.749011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:43.749011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:43.749011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:43.749011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:43.749011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:43.749011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.749011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:43.749702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:43.751017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.751017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:43.751017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.751017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:43.751790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:43.752953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:43.753799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.753799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:43.753799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.753799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:43.753799 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:43.753799 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:43.753799 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:43.753799 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:43.754136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:43.754136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:43.754136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.754136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:43.754136 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.754136 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:43.754136 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:43.754136 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.754136 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:43.754136 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.754136 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:43.754610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.754610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:43.754610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:43.754610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:43.754610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:43.754610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:43.754610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:43.754610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.754610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:43.754610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:43.754610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:43.754610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.755076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:43.755076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:43.755076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:43.755076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:43.755076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:43.755076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:43.755076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:43.755076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:43.755076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.755076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:43.755076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:43.755076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.755076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:43.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:43.756744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:43.758803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:43.758803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:43.758803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:43.758803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:43.758803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:43.758803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:45.738196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:45.738886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:45.739038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:45.740487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:45.778647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:45.843700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:45.846267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:45.847035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:45.847035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:45.847035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:45.847035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:45.847035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:45.847035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:45.847035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:45.847035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:45.847035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:45.847035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:45.847035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:45.847035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:45.847035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:45.849676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:45.855346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:45.855346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:45.855346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:45.855346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:45.855346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:45.855346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:45.855346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:45.855346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:45.855346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:45.855346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:45.855346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:45.855346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:48.722978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:48.723509 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:48.723509 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:48.724308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:48.724639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:48.724877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.725653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:48.725653 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.725653 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:48.725653 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:48.725653 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.725653 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:48.725653 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.725653 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:48.726384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.726384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:48.726384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:48.726384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:48.726384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:48.726384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:48.726384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:48.726384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.726384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:48.726384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:48.726384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:48.726384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.726648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:48.726648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:48.726648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:48.726648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:48.726648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:48.726648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:48.726648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:48.726648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:48.726648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.726648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:48.726648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:48.726648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.726648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:48.726896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.726896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:48.728317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:48.728781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.729173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:48.729173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.729173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:48.731413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:48.731876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:48.731876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:48.731876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:41:48.732386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:48.732429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.734323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:48.734865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.735233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:48.735233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:48.735233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:48.735233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:48.735233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:48.735233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:48.735233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:41:48.735233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:48.735233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:48.735233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.735233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:48.735233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:48.735233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.735501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:48.735772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:48.736068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:48.736464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:48.736464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:48.736464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.736464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:48.736464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:48.736464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.736464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:48.736464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.736464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:48.744762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:48.745098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:48.745145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:48.745288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:48.745793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:48.746157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.746712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:48.746712 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.746712 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:48.746712 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:48.746712 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.746712 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:48.746712 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.746712 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:48.747320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:48.747661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:48.747661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:48.747661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.747661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:48.747661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:48.747661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.747661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:48.747661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.747661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:48.748089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.748089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:48.748089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:48.748089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:48.748089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:48.748089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:48.748089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:48.748089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:48.748089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:48.748089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.748089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:48.748641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:48.749619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:48.750606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:48.750969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:48.750969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:48.750969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:48.750969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:48.750969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.750969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:48.750969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:48.750969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.750969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:48.750969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.750969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:48.751868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:48.752108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:48.752193 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:48.752312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:48.752839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:48.753146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:48.753350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.753928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:48.753928 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.753928 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:48.753928 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:48.753928 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.753928 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:48.753928 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.753928 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:48.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:48.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:48.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:48.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:48.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:48.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:48.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:48.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:48.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:48.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.755176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:48.755176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:48.755176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:48.755176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:48.755176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:48.755176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:48.755176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:48.755176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:48.755176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.755176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:48.755176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:48.755176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.755176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:48.755613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.755613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:48.755794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:48.756400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:48.756750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:48.756750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:48.756750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:50.741071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:50.741887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:50.741887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:50.742429 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:50.781025 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:50.781462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:50.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:50.782396 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:50.782396 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:50.782396 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:50.782396 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:50.782396 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:50.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:50.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:50.783115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:50.783115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:50.783115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:50.783115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:50.783115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:50.783115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:50.783115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:50.783115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:50.783115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:50.783115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:50.783115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:50.783115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:50.784432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:50.784432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:50.784432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:50.784432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:50.784432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:50.784432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:50.784432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:50.784432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:50.784432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:50.784432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:50.784432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:50.784432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:50.784432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:50.785002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:50.787123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:50.788617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:50.788617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:50.788617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:50.788617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:50.788617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:50.788617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:53.724879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:53.725525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:53.725578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:53.726793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:53.727092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:53.727347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.728182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:53.728182 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.728182 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:53.728182 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:53.728182 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.728182 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:53.728182 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.728182 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:53.729729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.729729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:53.729729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:53.729729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:53.729729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:53.729729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:53.729729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:53.729729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.729729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:53.729729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:53.729729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:53.729729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:53.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:53.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:53.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:53.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:53.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:53.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:53.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:53.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:53.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:53.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.730188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:53.730533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.730533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:53.731635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:53.732140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.732696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:53.732696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.732696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:53.732696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:53.732696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:53.732696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:53.732696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:41:53.733104 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:53.733398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:53.734155 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.734155 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:53.734155 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:53.734155 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.734155 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:53.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.734155 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:53.735788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:53.736295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:53.736295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:53.736295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.736295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:53.736295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:53.736295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.736295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:53.736295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.736295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:53.737278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.737278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:53.737278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:53.737278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:53.737278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:53.737278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:53.737278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:53.737278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:41:53.737278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:53.737278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:53.737278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.737278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:53.737713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:53.738107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.738107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:53.738107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.738107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:53.738994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:53.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:53.740002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.740002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:53.740002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:53.740002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.740002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:53.740002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.740002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:53.745598 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:53.745801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:53.745860 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:53.745964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:53.746489 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:53.746866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.747494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:53.747494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.747494 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:53.747494 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:53.747494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.747494 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:53.747494 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.747494 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:53.748007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:53.749493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:53.749938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:53.749938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:53.749938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:53.749938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:53.749938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.749938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:53.749938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:53.749938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:53.749938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.749938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:53.749938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:53.749938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:53.749938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:53.750328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:53.750328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:53.750328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:53.750328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:53.750328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.750328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:53.750328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:53.750328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.750328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:53.750328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.750328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:53.750328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:53.750670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.751101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:53.752257 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:53.752409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:53.752490 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:53.752601 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:53.752999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:53.753377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:53.753588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:53.754200 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.754200 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:53.754200 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:53.754200 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.754200 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:53.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:53.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:53.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:53.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:53.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:53.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:53.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:53.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:53.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:53.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:53.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.755045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:53.755045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:53.755045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:53.755045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:53.755045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:53.755045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:53.755045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:53.755045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:53.755045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.755045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:53.755045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:53.755045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.755045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:53.755303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.755303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:53.755541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:53.755998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:53.756449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:53.756449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:53.756449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:55.741861 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:55.742467 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:55.742543 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:55.743581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:55.786582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:55.786918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:55.787715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:55.787715 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:55.787715 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:55.787715 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:55.787715 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:55.787715 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:55.787715 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:55.787715 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:55.788636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:55.788636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:55.788636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:55.788636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:55.788636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:55.788636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:55.788636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:55.788636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:55.788636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:55.788636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:55.788636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:55.788636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:55.789067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:55.789067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:55.789067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:55.789067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:55.789067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:55.789067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:55.789067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:55.789067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:55.789067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:55.789067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:55.789067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:55.789067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:55.789067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:55.789527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:55.790007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:55.790609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:55.790609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:55.790609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:55.790609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:55.790609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:55.790609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:58.725897 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:58.726768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:58.726768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:58.727575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:58.727575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:58.727575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.728076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:58.728076 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.728076 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:58.728076 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:58.728076 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.728076 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:58.728076 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.728076 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:58.729348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.729348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:58.729348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:58.729348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:58.729348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:58.729348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:58.729348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:58.729348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.729348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:58.729348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:58.729348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:58.729348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.729958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:58.729958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:58.729958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:58.729958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:58.729958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:58.729958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:58.729958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:58.729958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:58.729958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.729958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:58.729958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:58.729958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.729958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:58.730857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:58.731532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.732347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:58.732347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:58.732347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.732347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:58.732347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.732347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:58.734062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:58.734173 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:58.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:58.734400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:41:58.735045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:58.735299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.735863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:58.735863 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.735863 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:58.735863 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:58.735863 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.735863 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:58.735863 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.735863 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:58.736909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:58.737294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:58.737294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:58.737294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.737294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:58.737294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:58.737294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.737294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:58.737294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.737294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:58.738156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.738156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:58.738156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:58.738156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:58.738156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:58.738156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:58.738156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:58.738156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:41:58.738156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:58.738156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:58.738156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.738156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:58.738472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:58.738754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.738754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:58.738754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.738754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:58.739928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:58.740479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:58.741117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.741117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:58.741117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:58.741117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.741117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:58.741117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.741117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:58.745986 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:58.746286 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:58.746566 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:58.746754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:58.747433 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:58.747803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.748543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:58.748543 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.748543 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:58.748543 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:58.748543 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.748543 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:58.748543 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.748543 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:41:58.749267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:41:58.749580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:58.749580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:41:58.749580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.749580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:41:58.749580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:41:58.749580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.749580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:41:58.749580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.749580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:41:58.750083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.750083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:58.750083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:58.750083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:58.750083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:58.750083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:58.750083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:58.750083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:58.750083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:58.750083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.750083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:58.750361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:58.750704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.750704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:58.750704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.750704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:58.751220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:58.751802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:58.754407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.754407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:58.754407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.754407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:41:58.754407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:41:58.754407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:41:58.754407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:41:58.754407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:41:58.754802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:58.754802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:41:58.754802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.754913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:41:58.754913 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.754913 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:41:58.754913 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:41:58.754913 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.754913 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:41:58.754913 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.754913 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:41:58.755691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.755691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:41:58.755691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:58.755691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:41:58.755691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:41:58.755691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:58.755691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:58.755691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.755691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:58.755691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:58.755691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:41:58.755691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.755993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:41:58.755993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:58.755993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:41:58.755993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:58.755993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:41:58.755993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:41:58.755993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:58.755993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:41:58.755993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.755993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:41:58.755993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:41:58.755993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.755993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:41:58.756273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.756273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:41:58.756615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:41:58.757113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:41:58.757542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:41:58.757542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:41:58.757542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:00.742472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:00.743408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:00.743408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:00.744328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:00.783171 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:00.783469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:00.784417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:00.784417 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:00.784417 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:00.784417 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:00.784417 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:00.784417 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:00.784417 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:00.784417 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:00.785486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:00.785486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:00.785486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:00.785486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:00.785486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:00.785486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:00.785486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:00.785486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:00.785486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:00.785486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:00.785486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:00.785486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:00.786200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:00.786200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:00.786200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:00.786200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:00.786200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:00.786200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:00.786200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:00.786200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:00.786200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:00.786200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:00.786200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:00.786200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:00.786200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:00.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:00.788013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:00.789495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:00.789495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:00.789495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:00.789495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:00.789495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:00.789495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:03.726134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:03.726595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:03.726595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:03.727226 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:03.727411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:03.727523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.728643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:03.728643 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.728643 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:03.728643 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:03.728643 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.728643 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:03.728643 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.728643 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:03.730313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.730313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:03.730313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:03.730313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:03.730313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:03.730313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:03.730313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:03.730313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.730313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:03.730313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:03.730313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:03.730313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.730603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:03.730603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:03.730603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:03.730603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:03.730603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:03.730603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:03.730603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:03.730603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:03.730603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.730603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:03.730603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:03.730603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.730603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:03.730873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.730873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:03.732186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:03.732570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.732951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:03.732951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.732951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:03.734287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:03.734524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:03.735169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:03.735169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:42:03.735169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:03.735389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.735911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:03.735911 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.735911 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:03.735911 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:03.735911 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.735911 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:03.735911 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.735911 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:03.736678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:03.737223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:03.737223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:03.737223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.737223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:03.737223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:03.737223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.737223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:03.737223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.737223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:03.737548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.737548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:03.737548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:03.737548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:03.737548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:03.737548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:03.737548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:03.737548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:42:03.737548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:03.737548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:03.737548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.737548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:03.737965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:03.738399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.738399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:03.738399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.738399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:03.738525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:03.738992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:03.739547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.739547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:03.739547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:03.739547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.739547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:03.739547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.739547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:03.746312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:03.746624 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:03.746808 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:03.746902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:03.747304 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:03.747732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.748181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:03.748181 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.748181 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:03.748181 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:03.748181 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.748181 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:03.748181 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.748181 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:03.749049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:03.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:03.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:03.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:03.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:03.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:03.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:03.750369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.750369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:03.750369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:03.750369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:03.750369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:03.750369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:03.750369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:03.750369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:03.750369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:03.750369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.750369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:03.750906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:03.751544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:03.752364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:03.753240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:03.753240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:03.753240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:03.753240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:03.753240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.753240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:03.753240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:03.753240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.753240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:03.753240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.753240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:03.753240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:03.753786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:03.753786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:03.753786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:03.754044 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:03.754196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:03.754325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.754722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:03.754722 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.754722 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:03.754722 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:03.754722 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.754722 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:03.754722 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.754722 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:03.755537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.755537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:03.755537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:03.755537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:03.755537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:03.755537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:03.755537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:03.755537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.755537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:03.755537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:03.755537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:03.755537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.756074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:03.756074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:03.756074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:03.756074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:03.756074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:03.756074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:03.756074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:03.756074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:03.756074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.756074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:03.756074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:03.756074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.756074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:03.756809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:03.757375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:03.757761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:03.757761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:03.757761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:03.757761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:03.757761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:03.757761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:05.743059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:05.744082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:05.744082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:05.745525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:05.746895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:42:05.779024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:05.779310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:05.779917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:05.779917 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:05.779917 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:05.779917 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:05.779917 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:05.779917 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:05.779917 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:05.779917 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:05.780742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:05.780742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:05.780742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:05.780742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:05.780742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:05.780742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:05.780742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:05.780742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:05.780742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:05.780742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:05.780742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:05.780742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:05.781426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:05.781426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:05.781426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:05.781426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:05.781426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:05.781426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:05.781426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:05.781426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:05.781426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:05.781426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:05.781426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:05.781426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:05.781426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:05.782084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:05.782863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:05.783841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:05.783841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:05.783841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:05.783841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:05.783841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:05.783841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:08.726739 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:08.727291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:08.727291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:08.728636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:08.728636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:08.728737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.729998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:08.729998 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.729998 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:08.729998 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:08.729998 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.729998 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:08.729998 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.729998 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:08.731170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.731170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:08.731170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:08.731170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:08.731170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:08.731170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:08.731170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:08.731170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.731170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:08.731170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:08.731170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:08.731170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.731507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:08.731507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:08.731507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:08.731507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:08.731507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:08.731507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:08.731507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:08.731507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:08.731507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.731507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:08.731507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:08.731507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.731507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:08.731803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:08.732157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.732585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:08.732585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:08.732585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.732585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:08.732585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.732585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:08.734689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:08.735027 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:08.735027 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:08.735150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:42:08.736736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:08.739245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.739900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:08.740483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.740483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:08.740483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:08.740483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:08.740483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:08.740483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:08.740483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:08.740483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:42:08.740483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:08.740483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:08.740483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.740483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:08.740877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:08.741383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:08.741840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:08.742480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:08.742480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:08.742480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:08.742480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:08.742480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:08.742480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.742480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:08.742480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:08.742480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.742480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:08.742480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.742480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:08.748073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:08.748310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:08.748421 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:08.748548 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:08.749032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:08.749359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.749996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:08.749996 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.749996 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:08.749996 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:08.749996 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.749996 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:08.749996 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.749996 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:08.750759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:08.751338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:08.751338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:08.751338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.751338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:08.751338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:08.751338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.751338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:08.751338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.751338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:08.751687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.751687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:08.751687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:08.751687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:08.751687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:08.751687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:08.751687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:08.751687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:08.751687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:08.751687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.751687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:08.752059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:08.752595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.752595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:08.752595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.752595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:08.752971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:08.753787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:08.754743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.754743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:08.754743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.754743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:08.754743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:08.754743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:08.754743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:08.754743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:08.755290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:08.755290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:08.755290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.755743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:08.755743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.755743 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:08.755743 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:08.755743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.755743 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:08.755743 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.755743 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:08.756991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.756991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:08.756991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:08.756991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:08.756991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:08.756991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:08.756991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:08.756991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.756991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:08.756991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:08.756991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:08.756991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:08.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:08.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:08.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:08.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:08.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:08.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:08.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:08.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:08.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:08.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:08.758135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.758135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:08.758513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:08.759580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:08.760516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:08.760516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:08.760516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:10.743753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:10.744229 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:10.744229 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:10.745176 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:10.782159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:10.782503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:10.782933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:10.782933 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:10.782933 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:10.782933 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:10.782933 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:10.782933 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:10.782933 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:10.782933 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:10.783698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:10.783698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:10.783698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:10.783698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:10.783698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:10.783698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:10.783698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:10.783698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:10.783698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:10.783698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:10.783698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:10.783698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:10.783987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:10.783987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:10.783987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:10.783987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:10.783987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:10.783987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:10.783987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:10.783987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:10.783987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:10.783987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:10.783987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:10.783987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:10.783987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:10.784254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:10.784254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:10.784483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:10.784798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:10.785134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:10.785134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:10.785134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:13.727398 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:13.727979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:13.728041 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:13.729302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:13.729755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:13.730079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.731334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:13.731334 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.731334 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:13.731334 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:13.731334 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.731334 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:13.731334 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.731334 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:13.732371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.732371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:13.732371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:13.732371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:13.732371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:13.732371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:13.732371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:13.732371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.732371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:13.732371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:13.732371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:13.732371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.732705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:13.732705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:13.732705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:13.732705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:13.732705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:13.732705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:13.732705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:13.732705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:13.732705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.732705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:13.732705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:13.732705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.732705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:13.732977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.732977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:13.733473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:13.733880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.734343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:13.734343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.734343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:13.735037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:13.735305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:13.735422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:13.735544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:42:13.735895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:13.736112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.736621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:13.736621 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.736621 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:13.736621 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:13.736621 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.736621 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:13.736621 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.736621 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:13.737422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:13.737908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:13.737908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:13.737908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.737908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:13.737908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:13.737908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.737908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:13.737908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.737908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:13.738449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.738449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:13.738449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:13.738449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:13.738449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:13.738449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:13.738449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:13.738449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:42:13.738449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:13.738449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:13.738449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.738449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:13.738799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:13.739161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.739161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:13.739161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.739161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:13.739527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:13.739995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:13.740487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.740487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:13.740487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:13.740487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.740487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:13.740487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.740487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:13.748385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:13.748748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:13.748808 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:13.748996 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:13.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:13.750040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.750612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:13.750612 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.750612 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:13.750612 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:13.750612 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.750612 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:13.750612 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.750612 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:13.751408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:13.752187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:13.752187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:13.752187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.752187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:13.752187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:13.752187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.752187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:13.752187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.752187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:13.752774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.752774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:13.752774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:13.752774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:13.752774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:13.752774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:13.752774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:13.752774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:13.752774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:13.752774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.752774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:13.753550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:13.754120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:13.754754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:13.755684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:13.755684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:13.755684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:13.755684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:13.755684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.755684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:13.755684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:13.755684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.755684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:13.755684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.755684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:13.755684 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:13.755684 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:13.756087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:13.756087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:13.756087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:13.756087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:13.756087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.756087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:13.756087 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.756087 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:13.756087 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:13.756374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:13.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:13.757342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:13.757801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:13.758546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:13.758546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:13.758546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:13.758546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:13.758546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:13.758546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:15.744179 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:15.744823 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:15.744935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:15.746402 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:15.784020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:15.784303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:15.784949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:15.784949 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:15.784949 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:15.784949 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:15.784949 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:15.784949 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:15.784949 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:15.784949 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:15.785754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:15.785754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:15.785754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:15.785754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:15.785754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:15.785754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:15.785754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:15.785754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:15.785754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:15.785754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:15.785754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:15.785754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:15.786177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:15.786177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:15.786177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:15.786177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:15.786177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:15.786177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:15.786177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:15.786177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:15.786177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:15.786177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:15.786177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:15.786177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:15.786177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:15.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:15.787029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:15.787560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:15.787560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:15.787560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:15.787560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:15.787560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:15.787560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:18.728181 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:18.729107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:18.729107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:18.729881 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:18.730201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:18.730473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.731280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:18.731280 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.731280 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:18.731280 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:18.731280 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.731280 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:18.731280 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.731280 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:18.732268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.732268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:18.732268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:18.732268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:18.732268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:18.732268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:18.732268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:18.732268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.732268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:18.732268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:18.732268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:18.732268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.732872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:18.732872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:18.732872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:18.732872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:18.732872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:18.732872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:18.732872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:18.732872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:18.732872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.732872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:18.732872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:18.732872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.732872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:18.733562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:18.734296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.735155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:18.735155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:18.735155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.735155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:18.735155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.735155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:18.735478 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:18.735787 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:18.735861 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:18.736005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:42:18.736371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:18.736563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.737068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:18.737068 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.737068 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:18.737068 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:18.737068 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.737068 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:18.737068 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.737068 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:18.737803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:18.738279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:18.738279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:18.738279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.738279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:18.738279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:18.738279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.738279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:18.738279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.738279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:18.738556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.738556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:18.738556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:18.738556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:18.738556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:18.738556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:18.738556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:18.738556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:42:18.738556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:18.738556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:18.738556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.738556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:18.738910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:18.739312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.739312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:18.739312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.739312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:18.739556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:18.739993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:18.740531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.740531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:18.740531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:18.740531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.740531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:18.740531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.740531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:18.748838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:18.749883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:18.750099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:18.750320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:18.750764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:18.751022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.751654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:18.751654 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.751654 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:18.751654 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:18.751654 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.751654 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:18.751654 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.751654 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:18.752439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:18.753174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:18.753174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:18.753174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.753174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:18.753174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:18.753174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.753174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:18.753174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.753174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:18.753623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.753623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:18.753623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:18.753623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:18.753623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:18.753623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:18.753623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:18.753623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:18.753623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:18.753623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.753623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:18.754200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:18.754876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:18.755579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:18.756502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:18.756502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:18.756502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:18.756502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:18.756502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.756502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:18.756502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:18.756502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.756502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:18.756502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.756502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:18.756502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:18.756502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:18.757193 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:18.757193 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:18.757193 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:18.757193 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:18.757193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.757193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:18.757193 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.757193 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:18.757193 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:18.757713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:18.758130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:18.758531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:18.758969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:18.759504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:18.759504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:18.759504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:18.759504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:18.759504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:18.759504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:20.745686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:20.745686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:20.745686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:20.747197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:20.783383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:20.783710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:20.784469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:20.784469 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:20.784469 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:20.784469 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:20.784469 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:20.784469 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:20.784469 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:20.784469 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:20.785751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:20.785751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:20.785751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:20.785751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:20.785751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:20.785751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:20.785751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:20.785751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:20.785751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:20.785751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:20.785751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:20.785751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:20.786333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:20.786333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:20.786333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:20.786333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:20.786333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:20.786333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:20.786333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:20.786333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:20.786333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:20.786333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:20.786333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:20.786333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:20.786333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:20.786836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:20.787424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:20.788768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:20.788768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:20.788768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:20.788768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:20.788768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:20.788768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:23.730869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:23.731833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:23.731931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:23.732846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:23.733034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:23.733858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.733858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:23.733858 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.733858 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:23.733858 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:23.733858 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.733858 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:23.733858 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.733858 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:23.734677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.734677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:23.734677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:23.734677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:23.734677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:23.734677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:23.734677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:23.734677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.734677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:23.734677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:23.734677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:23.734677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.734998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:23.734998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:23.734998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:23.734998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:23.734998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:23.734998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:23.734998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:23.734998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:23.734998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.734998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:23.734998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:23.734998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.734998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:23.735293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.735293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:23.736261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:23.737009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.737454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:23.737454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.737454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:23.737454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:23.737454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:23.737454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:23.737454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:42:23.737772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:23.737946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.739015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:23.739015 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.739015 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:23.739015 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:23.739015 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.739015 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:23.739015 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.739015 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:23.739314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:23.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:23.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:23.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:23.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:23.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:23.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:23.740512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.740512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:23.740512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:23.740512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:23.740512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:23.740512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:23.740512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:23.740512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:42:23.740512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:23.740512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:23.740512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.740512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:23.740812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:23.741068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.741068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:23.741068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.741068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:23.741529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:23.741880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:23.742255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.742255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:23.742255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:23.742255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.742255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:23.742255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.742255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:23.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:23.751947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:23.752019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:23.752146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:23.752801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:23.753068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.753859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:23.753859 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.753859 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:23.753859 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:23.753859 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.753859 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:23.753859 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.753859 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:23.756054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:23.756541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:23.756541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:23.756541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.756541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:23.756541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:23.756541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.756541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:23.756541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.756541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:23.758011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.758011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:23.758011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:23.758011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:23.758011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:23.758011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:23.758011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:23.758011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:23.758011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:23.758011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.758011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:23.758397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:23.758669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.758669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:23.758669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.758669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:23.758669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:23.760646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:23.761040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:23.761443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.761443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:23.761443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.761443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:23.761443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:23.761443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:23.761721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:23.762367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:23.762514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:23.762633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.763434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:23.763434 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.763434 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:23.763434 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:23.763434 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.763434 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:23.763434 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.763434 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:23.764404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.764404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:23.764404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:23.764404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:23.764404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:23.764404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:23.764404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:23.764404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.764404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:23.764404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:23.764404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:23.764404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.764897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:23.764897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:23.764897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:23.764897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:23.764897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:23.764897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:23.764897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:23.764897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:23.764897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.764897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:23.764897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:23.764897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.764897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:23.765129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.765129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:23.767348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:23.767846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:23.768191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:23.768191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:23.768191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:25.745545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:25.746446 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:25.746446 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:25.747047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:25.782519 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:25.782768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:25.783331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:25.783331 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:25.783331 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:25.783331 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:25.783331 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:25.783331 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:25.783331 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:25.783331 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:25.784394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:25.784394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:25.784394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:25.784394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:25.784394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:25.784394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:25.784394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:25.784394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:25.784394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:25.784394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:25.784394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:25.784394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:25.785021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:25.785021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:25.785021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:25.785021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:25.785021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:25.785021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:25.785021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:25.785021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:25.785021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:25.785021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:25.785021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:25.785021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:25.785021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:25.785604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:25.787568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:25.789824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:25.789824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:25.789824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:25.789824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:25.789824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:25.789824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:28.731994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:28.732722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:28.732722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:28.734089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:28.734196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:28.734502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.735446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:28.735446 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.735446 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:28.735446 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:28.735446 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.735446 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:28.735446 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.735446 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:28.737012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.737012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:28.737012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:28.737012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:28.737012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:28.737012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:28.737012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:28.737012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.737012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:28.737012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:28.737012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:28.737012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.737524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:28.737524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:28.737524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:28.737524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:28.737524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:28.737524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:28.737524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:28.737524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:28.737524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.737524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:28.737524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:28.737524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.737524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:28.737953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.737953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:28.739351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:28.740090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.741544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:28.741544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.741544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:28.741544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:28.741544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:28.741544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:28.741544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:42:28.741544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:28.741848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.741992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:28.741992 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.741992 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:28.741992 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:28.741992 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.741992 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:28.741992 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.741992 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:28.743412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:28.743964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:28.743964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:28.743964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.743964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:28.743964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:28.743964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.743964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:28.743964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.743964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:28.744813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.744813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:28.744813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:28.744813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:28.744813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:28.744813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:28.744813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:28.744813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:42:28.744813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:28.744813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:28.744813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.744813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:28.745096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:28.745362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.745362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:28.745362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.745362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:28.747133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:28.747537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:28.747912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.747912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:28.747912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:28.747912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.747912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:28.747912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.747912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:28.752299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:28.752299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:28.752386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:28.752513 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:28.756241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.756704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:28.757226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.757226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:28.757226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.757226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:28.757226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:28.757226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:28.757226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:28.757226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:28.757226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:28.757226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:28.757226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:28.757226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.757612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:28.758007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:28.758475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:28.759006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:28.759006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:28.759006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:28.759006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:28.759006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.759006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:28.759006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:28.759006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.759006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:28.759006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.759006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:28.761294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:28.761652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:28.761778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:28.761975 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:28.764856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:28.764856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:28.764856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.764856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:28.764856 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.764856 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:28.764856 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:28.764856 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.764856 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:28.764856 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.764856 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:28.764856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:28.765148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:28.765566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:28.765566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:28.765566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:28.765566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:28.765566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:28.765566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.765566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:28.765566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:28.765566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.765566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:28.765566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.765566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:28.765810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:28.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:28.766526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:28.766526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:28.766526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:30.746068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:30.746666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:30.746666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:30.747827 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:30.777178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:30.777569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:30.778300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:30.778300 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:30.778300 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:30.778300 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:30.778300 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:30.778300 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:30.778300 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:30.778300 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:30.780098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:30.780098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:30.780098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:30.780098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:30.780098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:30.780098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:30.780098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:30.780098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:30.780098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:30.780098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:30.780098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:30.780098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:30.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:30.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:30.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:30.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:30.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:30.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:30.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:30.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:30.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:30.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:30.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:30.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:30.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:30.781101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:30.781101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:30.782007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:30.782707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:30.783294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:30.783294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:30.783294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:33.732914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:33.733876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:33.733876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:33.734964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:33.735247 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:33.735419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.736472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:33.736472 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.736472 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:33.736472 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:33.736472 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.736472 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:33.736472 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.736472 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:33.737992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.737992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:33.737992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:33.737992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:33.737992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:33.737992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:33.737992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:33.737992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.737992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:33.737992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:33.737992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:33.737992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.738621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:33.738621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:33.738621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:33.738621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:33.738621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:33.738621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:33.738621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:33.738621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:33.738621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.738621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:33.738621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:33.738621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.738621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:33.739177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.739177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:33.740284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:33.741738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.742612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:33.742612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.742612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:33.742612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:33.742612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:33.742612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:33.742612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:42:33.742612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:33.743012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.743012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:33.743012 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.743012 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:33.743012 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:33.743012 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.743012 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:33.743012 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.743012 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:33.743824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:33.744299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:33.744299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:33.744299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.744299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:33.744299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:33.744299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.744299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:33.744299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.744299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:33.744592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.744592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:33.744592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:33.744592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:33.744592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:33.744592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:33.744592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:33.744592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:42:33.744592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:33.744592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:33.744592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.744592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:33.744977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:33.746586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:33.746879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:33.747224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:33.747224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:33.747224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:33.747224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:33.747224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:33.747224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.747224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:33.747224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:33.747224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.747224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:33.747224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.747224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:33.752462 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:33.752905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:33.753113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:33.753366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:33.753788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:33.754113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.756177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.756694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:33.757732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.757732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:33.757732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:33.757732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:33.757732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:33.757732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:33.757732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:33.757732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:33.757732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:33.757732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.757732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:33.758113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:33.759002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.759002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:33.759002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.759002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:33.759513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:33.759986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:33.760465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.760465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:33.760465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.760465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:33.761842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:33.762241 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:33.762310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:33.762404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:33.762833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:33.763157 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:33.763520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.764319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:33.764319 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.764319 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:33.764319 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:33.764319 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.764319 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:33.764319 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.764319 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:33.764919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.764919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:33.764919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:33.764919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:33.764919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:33.764919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:33.764919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:33.764919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.764919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:33.764919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:33.764919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:33.764919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.765177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:33.765177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:33.765177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:33.765177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:33.765177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:33.765177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:33.765177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:33.765177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:33.765177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.765177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:33.765177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:33.765177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.765177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:33.765454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.765454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:33.765615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:33.765913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:33.766281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:33.766281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:33.766281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:35.746921 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:35.748882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:35.748882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:35.749042 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:35.782577 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:35.782890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:35.783558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:35.783558 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:35.783558 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:35.783558 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:35.783558 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:35.783558 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:35.783558 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:35.783558 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:35.784397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:35.784397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:35.784397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:35.784397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:35.784397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:35.784397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:35.784397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:35.784397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:35.784397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:35.784397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:35.784397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:35.784397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:35.784689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:35.784689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:35.784689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:35.784689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:35.784689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:35.784689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:35.784689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:35.784689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:35.784689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:35.784689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:35.784689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:35.784689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:35.784689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:35.784945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:35.784945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:35.785332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:35.786369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:35.786741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:35.786741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:35.786741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:38.733694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:38.734404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:38.734404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:38.735414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:38.736289 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:38.736289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.736652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:38.736652 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.736652 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:38.736652 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:38.736652 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.736652 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:38.736652 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.736652 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:38.737440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.737440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:38.737440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:38.737440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:38.737440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:38.737440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:38.737440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:38.737440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.737440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:38.737440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:38.737440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:38.737440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.737726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:38.737726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:38.737726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:38.737726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:38.737726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:38.737726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:38.737726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:38.737726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:38.737726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.737726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:38.737726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:38.737726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.737726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:38.737993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.737993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:38.738508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:38.739877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.740637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:38.740637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.740637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:38.741277 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:38.741512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:38.741821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:38.741937 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:42:38.742438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:38.742961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.743669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:38.743669 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.743669 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:38.743669 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:38.743669 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.743669 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:38.743669 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.743669 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:38.744380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:38.745952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:38.747334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:38.747334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:38.747334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:42:38.747334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:38.747334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:38.747334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.747334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:38.747334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:38.747334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:38.747334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.747334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:38.747334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:38.747334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:38.748048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:38.748048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:38.748048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:38.748048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:38.748048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:38.748048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.748048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:38.748048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:38.748048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.748048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:38.748048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.748048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:38.748611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:38.749304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:38.749739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.749739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:38.749739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:38.749739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.749739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:38.749739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.749739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:38.752901 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:38.753200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:38.753338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:38.753475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:38.754599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:38.754599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:38.754750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.754750 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:38.754750 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:38.754750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.754750 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:38.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.754750 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:38.755609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:38.755999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:38.755999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:38.755999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.755999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:38.755999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:38.755999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.755999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:38.755999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.755999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:38.756582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.756582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:38.756582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:38.756582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:38.756582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:38.756582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:38.756582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:38.756582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:38.756582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:38.756582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.756582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:38.757194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:38.757570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.757570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:38.757570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.757570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:38.757677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:38.758111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:38.758606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.758606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:38.758606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.758606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:38.762396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:38.762664 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:38.762782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:38.762952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:38.763718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:38.764076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:38.764763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.764971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:38.764971 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.764971 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:38.764971 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:38.764971 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.764971 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:38.764971 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.764971 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:38.765843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.765843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:38.765843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:38.765843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:38.765843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:38.765843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:38.765843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:38.765843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.765843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:38.765843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:38.765843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:38.765843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.766370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:38.766370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:38.766370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:38.766370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:38.766370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:38.766370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:38.766370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:38.766370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:38.766370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.766370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:38.766370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:38.766370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.766370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:38.766827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.766827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:38.766904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:38.767292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:38.767640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:38.767640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:38.767640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:40.751119 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:40.751884 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:40.751884 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:40.752563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:40.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:40.781294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:40.781716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:40.781716 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:40.781716 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:40.781716 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:40.781716 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:40.781716 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:40.781716 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:40.781716 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:40.782557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:40.782557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:40.782557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:40.782557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:40.782557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:40.782557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:40.782557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:40.782557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:40.782557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:40.782557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:40.782557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:40.782557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:40.783281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:40.783281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:40.783281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:40.783281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:40.783281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:40.783281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:40.783281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:40.783281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:40.783281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:40.783281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:40.783281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:40.783281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:40.783281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:40.784502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:40.785284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:40.786131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:40.786131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:40.786131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:40.786131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:40.786131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:40.786131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:43.734517 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:43.736059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:43.736059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:43.736482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:43.736804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:43.737040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.738770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:43.738770 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.738770 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:43.738770 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:43.738770 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.738770 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:43.738770 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.738770 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:43.740452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.740452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:43.740452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:43.740452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:43.740452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:43.740452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:43.740452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:43.740452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.740452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:43.740452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:43.740452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:43.740452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.740824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:43.740824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:43.740824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:43.740824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:43.740824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:43.740824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:43.740824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:43.740824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:43.740824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.740824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:43.740824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:43.740824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.740824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:43.741087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.741087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:43.742578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:43.743076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.743622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:43.743622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.743622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:43.744393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:43.744866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:43.748264 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:43.748264 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:42:43.748264 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:43.748264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.748264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:43.748264 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.748264 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:43.748264 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:43.748264 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.748264 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:43.748264 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.748264 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:43.748628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:43.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:43.749398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:43.749398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:43.749398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:42:43.749398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:43.749398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:43.749398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.749398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:43.749398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:43.749398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:43.749398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.749398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:43.749398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:43.749398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:43.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:43.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:43.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:43.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:43.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:43.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:43.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:43.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:43.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.749746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:43.750053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:43.750515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:43.750957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.750957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:43.750957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:43.750957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.750957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:43.750957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.750957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:43.753194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:43.753377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:43.753439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:43.753551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:43.754000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:43.754276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.755598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:43.755598 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.755598 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:43.755598 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:43.755598 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.755598 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:43.755598 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.755598 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:43.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:43.756117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:43.756117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:43.756117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.756117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:43.756117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:43.756117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.756117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:43.756117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.756117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:43.756365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.756365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:43.756365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:43.756365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:43.756365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:43.756365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:43.756365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:43.756365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:43.756365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:43.756365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.756365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:43.756627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:43.756903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.756903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:43.756903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.756903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:43.757323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:43.757865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:43.758278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.758278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:43.758278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.758278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:43.762762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:43.762964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:43.763035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:43.763728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:43.763978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:43.764288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:43.764447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.765020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:43.765020 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.765020 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:43.765020 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:43.765020 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.765020 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:43.765020 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.765020 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:43.765663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.765663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:43.765663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:43.765663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:43.765663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:43.765663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:43.765663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:43.765663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.765663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:43.765663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:43.765663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:43.765663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.766027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:43.766027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:43.766027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:43.766027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:43.766027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:43.766027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:43.766027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:43.766027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:43.766027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.766027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:43.766027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:43.766027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.766027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:43.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:43.766805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:43.767251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:43.767251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:43.767251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:43.767251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:43.767251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:43.767251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:45.751652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:45.752130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:45.752130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:45.752829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:45.785683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:45.786002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:45.786729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:45.786729 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:45.786729 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:45.786729 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:45.786729 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:45.786729 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:45.786729 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:45.786729 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:45.787587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:45.787587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:45.787587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:45.787587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:45.787587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:45.787587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:45.787587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:45.787587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:45.787587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:45.787587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:45.787587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:45.787587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:45.788006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:45.788006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:45.788006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:45.788006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:45.788006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:45.788006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:45.788006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:45.788006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:45.788006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:45.788006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:45.788006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:45.788006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:45.788006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:45.788388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:45.788845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:45.789376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:45.789376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:45.789376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:45.789376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:45.789376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:45.789376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:48.735117 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:48.735871 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:48.735871 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:48.738306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:48.738306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:48.738306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.738306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:48.738306 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.738306 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:48.738306 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:48.738306 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.738306 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:48.738306 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.738306 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:48.738721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.738721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:48.738721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:48.738721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:48.738721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:48.738721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:48.738721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:48.738721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.738721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:48.738721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:48.738721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:48.738721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.739083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:48.739083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:48.739083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:48.739083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:48.739083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:48.739083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:48.739083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:48.739083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:48.739083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.739083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:48.739083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:48.739083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.739083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:48.739480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:48.739925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.740519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:48.740519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:48.740519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.740519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:48.740519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.740519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:48.745032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:48.745441 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:48.745596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:48.745816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:42:48.746114 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:48.746289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.746716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:48.746716 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.746716 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:48.746716 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:48.746716 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.746716 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:48.746716 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.746716 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:48.747915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:48.748338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:48.748338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:48.748338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.748338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:48.748338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:48.748338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.748338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:48.748338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.748338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:48.748975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.748975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:48.748975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:48.748975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:48.748975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:48.748975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:48.748975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:48.748975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:42:48.748975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:48.748975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:48.748975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.748975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:48.749284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:48.749548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.749548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:48.749548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.749548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:48.750392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:48.750750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:48.751095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.751095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:48.751095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:48.751095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.751095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:48.751095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.751095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:48.753525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:48.753783 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:48.753893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:48.754027 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:48.754340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:48.754504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.754919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:48.754919 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.754919 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:48.754919 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:48.754919 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.754919 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:48.754919 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.754919 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:48.755800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:48.756092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:48.756092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:48.756092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.756092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:48.756092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:48.756092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.756092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:48.756092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.756092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:48.756713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.756713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:48.756713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:48.756713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:48.756713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:48.756713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:48.756713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:48.756713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:48.756713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:48.756713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.756713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:48.756966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:48.757245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.757245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:48.757245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.757245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:48.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:48.758408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:48.758741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.758741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:48.758741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.758741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:48.763083 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:48.763323 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:48.763434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:48.763602 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:48.763961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:48.765343 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:48.765343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.765343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:48.765343 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.765343 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:48.765343 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:48.765343 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.765343 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:48.765343 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.765343 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:48.766178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.766178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:48.766178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:48.766178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:48.766178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:48.766178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:48.766178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:48.766178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.766178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:48.766178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:48.766178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:48.766178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.766819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:48.766819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:48.766819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:48.766819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:48.766819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:48.766819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:48.766819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:48.766819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:48.766819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.766819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:48.766819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:48.766819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.766819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:48.767069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:48.767373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:48.767764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:48.767764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:48.767764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:48.767764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:48.767764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:48.767764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:50.752158 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:50.752711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:50.752711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:50.753165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:50.780284 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:50.780284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:50.781064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:50.781064 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:50.781064 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:50.781064 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:50.781064 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:50.781064 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:50.781064 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:50.781064 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:50.781863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:50.781863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:50.781863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:50.781863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:50.781863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:50.781863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:50.781863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:50.781863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:50.781863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:50.781863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:50.781863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:50.781863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:50.782153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:50.782153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:50.782153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:50.782153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:50.782153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:50.782153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:50.782153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:50.782153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:50.782153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:50.782153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:50.782153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:50.782153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:50.782153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:50.782427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:50.782427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:50.786299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:50.787097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:50.787505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:50.787505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:50.787505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:53.735722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:53.736423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:53.736423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:53.736877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:53.736877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:53.737001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.738932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:53.738932 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.738932 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:53.738932 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:53.738932 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.738932 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:53.738932 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.738932 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:53.739229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.739229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:53.739229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:53.739229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:53.739229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:53.739229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:53.739229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:53.739229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.739229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:53.739229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:53.739229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:53.739229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.739587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:53.739587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:53.739587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:53.739587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:53.739587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:53.739587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:53.739587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:53.739587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:53.739587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.739587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:53.739587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:53.739587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.739587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:53.739979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.739979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:53.740183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:53.740664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.741233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:53.741233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.741233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:53.745452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:53.745622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:53.745697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:53.745785 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:42:53.746150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:53.746379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.746937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:53.746937 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.746937 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:53.746937 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:53.746937 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.746937 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:53.746937 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.746937 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:53.748672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:53.748999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:53.749299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:53.749299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:53.749299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:42:53.749299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:53.749299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:53.749299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.749299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:53.749299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:53.749299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:53.749299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.749299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:53.749299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:53.749299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:53.749546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:53.749546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:53.749546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:53.749546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:53.749546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:53.749546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.749546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:53.749546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:53.749546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.749546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:53.749546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.749546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:53.749785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:53.750079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:53.750429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.750429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:53.750429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:53.750429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.750429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:53.750429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.750429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:53.754084 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:53.754355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:53.754462 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:53.754594 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:53.755535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:53.757011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.757011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:53.757011 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.757011 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:53.757011 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:53.757011 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.757011 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:53.757011 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.757011 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:53.757314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:53.757712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:53.757712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:53.757712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.757712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:53.757712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:53.757712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.757712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:53.757712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.757712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:53.758127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.758127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:53.758127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:53.758127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:53.758127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:53.758127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:53.758127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:53.758127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:53.758127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:53.758127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.758127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:53.758501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:53.758877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.758877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:53.758877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.758877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:53.759220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:53.759683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:53.760254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.760254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:53.760254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.760254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:53.763347 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:53.764614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:53.764743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:53.764939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:53.765357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:53.765481 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:53.765610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:53.766130 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.766130 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:53.766130 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:53.766130 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.766130 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:53.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.766130 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:53.767108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.767108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:53.767108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:53.767108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:53.767108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:53.767108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:53.767108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:53.767108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.767108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:53.767108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:53.767108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:53.767108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.767411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:53.767411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:53.767411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:53.767411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:53.767411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:53.767411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:53.767411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:53.767411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:53.767411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.767411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:53.767411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:53.767411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.767411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:53.767669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.767669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:53.768588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:53.768915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:53.769432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:53.769432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:53.769432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:55.752905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:55.753508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:55.753508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:55.754155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:55.786545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:55.786743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:55.787389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:55.787389 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:55.787389 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:55.787389 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:55.787389 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:55.787389 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:55.787389 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:55.787389 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:55.789365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:55.789365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:55.789365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:55.789365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:55.789365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:55.789365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:55.789365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:55.789365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:55.789365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:55.789365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:55.789365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:55.789365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:55.790068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:55.790068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:55.790068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:55.790068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:55.790068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:55.790068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:55.790068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:55.790068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:55.790068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:55.790068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:55.790068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:55.790068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:55.790068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:55.790666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:55.791292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:55.792101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:55.792101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:55.792101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:55.792101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:55.792101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:55.792101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:58.736237 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:58.737238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:58.737340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:58.738070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:58.738298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:58.738427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.738897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:58.738897 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.738897 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:58.738897 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:58.738897 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.738897 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:58.738897 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.738897 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:58.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:58.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:58.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:58.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:58.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:58.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:58.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:58.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:58.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:58.739663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.740027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:58.740027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:58.740027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:58.740027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:58.740027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:58.740027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:58.740027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:58.740027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:58.740027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.740027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:58.740027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:58.740027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.740027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:58.740398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.740398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:58.740461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:58.740894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.741408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:58.741408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.741408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:58.746022 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:58.746491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:58.746564 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:58.746668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:42:58.747268 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:58.747492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.748090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:58.748090 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.748090 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:58.748090 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:58.748090 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.748090 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:58.748090 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.748090 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:58.748696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:58.749106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:58.749106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:58.749106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.749106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:58.749106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:58.749106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.749106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:58.749106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.749106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:58.749404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.749404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:58.749404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:58.749404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:58.749404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:58.749404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:58.749404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:58.749404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:42:58.749404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:58.749404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:58.749404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.749404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:58.749717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:58.750080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:58.751456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:58.752076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:58.752076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:58.752076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:58.752076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:58.752076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:58.752076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.752076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:58.752076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:58.752076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.752076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:58.752076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.752076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:58.754439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:58.754761 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:58.754868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:58.754975 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:58.755502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:58.755792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.756357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:58.756357 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.756357 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:58.756357 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:58.756357 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.756357 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:58.756357 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.756357 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:42:58.756952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:42:58.757271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:58.757271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:42:58.757271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.757271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:42:58.757271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:42:58.757271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.757271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:42:58.757271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.757271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:42:58.757680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.757680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:58.757680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:58.757680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:58.757680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:58.757680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:58.757680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:58.757680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:58.757680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:58.757680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.757680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:58.757914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:58.758182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.758182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:58.758182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.758182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:58.758492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:58.758810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:58.759136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.759136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:58.759136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.759136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:42:58.765290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:42:58.765290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:42:58.765290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:42:58.765504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:42:58.766114 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:58.766489 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:42:58.766694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.767414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:42:58.767414 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.767414 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:42:58.767414 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:42:58.767414 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.767414 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:42:58.767414 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.767414 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:42:58.768125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.768125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:42:58.768125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:58.768125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:42:58.768125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:42:58.768125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:58.768125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:58.768125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.768125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:58.768125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:58.768125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:42:58.768125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.768517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:42:58.768517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:58.768517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:42:58.768517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:58.768517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:42:58.768517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:42:58.768517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:58.768517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:42:58.768517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.768517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:42:58.768517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:42:58.768517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.768517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:42:58.768857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.768857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:42:58.769163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:42:58.769635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:42:58.770269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:42:58.770269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:42:58.770269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:00.753529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:00.754327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:00.754327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:00.755402 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:00.786758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:00.787012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:00.787677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:00.787677 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:00.787677 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:00.787677 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:00.787677 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:00.787677 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:00.787677 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:00.787677 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:00.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:00.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:00.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:00.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:00.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:00.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:00.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:00.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:00.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:00.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:00.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:00.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:00.789003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:00.789003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:00.789003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:00.789003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:00.789003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:00.789003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:00.789003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:00.789003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:00.789003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:00.789003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:00.789003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:00.789003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:00.789003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:00.789369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:00.789369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:00.789669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:00.790104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:00.790637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:00.790637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:00.790637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:03.737687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:03.738329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:03.738329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:03.740159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:03.740523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:03.740769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.741663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:03.741663 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.741663 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:03.741663 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:03.741663 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.741663 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:03.741663 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.741663 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:03.743119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.743119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:03.743119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:03.743119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:03.743119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:03.743119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:03.743119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:03.743119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.743119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:03.743119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:03.743119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:03.743119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.743543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:03.743543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:03.743543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:03.743543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:03.743543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:03.743543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:03.743543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:03.743543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:03.743543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.743543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:03.743543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:03.743543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.743543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:03.743918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.743918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:03.744794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:03.745355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.745992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:03.745992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.745992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:03.746540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:03.746910 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:03.747017 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:03.747131 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:43:03.747615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:03.747853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.748473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:03.748473 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.748473 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:03.748473 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:03.748473 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.748473 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:03.748473 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.748473 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:03.749330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:03.749846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:03.749846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:03.749846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.749846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:03.749846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:03.749846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.749846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:03.749846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.749846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:03.750278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.750278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:03.750278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:03.750278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:03.750278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:03.750278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:03.750278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:03.750278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:43:03.750278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:03.750278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:03.750278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.750278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:03.750749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:03.751153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.751153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:03.751153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.751153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:03.751295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:03.751776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:03.752355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.752355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:03.752355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:03.752355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.752355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:03.752355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.752355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:03.754887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:03.755183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:03.755312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:03.755431 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:03.755837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:03.756176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.756902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:03.756902 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.756902 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:03.756902 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:03.756902 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.756902 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:03.756902 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.756902 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:03.757581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:03.758014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:03.758014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:03.758014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.758014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:03.758014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:03.758014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.758014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:03.758014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.758014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:03.758418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.758418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:03.758418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:03.758418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:03.758418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:03.758418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:03.758418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:03.758418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:03.758418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:03.758418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.758418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:03.758779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:03.759178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.759178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:03.759178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.759178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:03.759433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:03.759983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:03.760574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.760574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:03.760574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.760574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:03.764903 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:03.765140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:03.765267 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:03.765430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:03.765813 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:03.766131 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:03.766376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.767008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:03.767008 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.767008 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:03.767008 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:03.767008 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.767008 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:03.767008 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.767008 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:03.767795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.767795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:03.767795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:03.767795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:03.767795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:03.767795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:03.767795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:03.767795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.767795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:03.767795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:03.767795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:03.767795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.768221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:03.768221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:03.768221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:03.768221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:03.768221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:03.768221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:03.768221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:03.768221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:03.768221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.768221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:03.768221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:03.768221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.768221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:03.768597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.768597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:03.768726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:03.769189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:03.769812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:03.769812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:03.769812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:05.754037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:05.754971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:05.754971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:05.755351 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:05.758788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:43:05.795125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:05.795449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:05.796144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:05.796144 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:05.796144 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:05.796144 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:05.796144 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:05.796144 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:05.796144 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:05.796144 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:05.796829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:05.796829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:05.796829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:05.796829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:05.796829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:05.796829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:05.796829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:05.796829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:05.796829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:05.796829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:05.796829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:05.796829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:05.797301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:05.797301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:05.797301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:05.797301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:05.797301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:05.797301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:05.797301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:05.797301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:05.797301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:05.797301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:05.797301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:05.797301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:05.797301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:05.797684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:05.797684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:05.797760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:05.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:05.798854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:05.798854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:05.798854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:08.738200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:08.739103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:08.739103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:08.740244 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:08.740592 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:08.741046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.741981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:08.741981 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.741981 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:08.741981 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:08.741981 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.741981 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:08.741981 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.741981 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:08.743409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.743409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:08.743409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:08.743409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:08.743409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:08.743409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:08.743409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:08.743409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.743409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:08.743409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:08.743409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:08.743409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.743990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:08.743990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:08.743990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:08.743990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:08.743990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:08.743990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:08.743990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:08.743990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:08.743990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.743990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:08.743990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:08.743990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.743990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:08.744559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.744559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:08.745007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:08.745821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.747359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:08.747359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.747359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:08.749522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:08.749522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:08.749522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:08.749522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:43:08.749878 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:08.749961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.751072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:08.751072 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.751072 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:08.751072 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:08.751072 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.751072 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:08.751072 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.751072 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:08.752665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:08.753341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:08.753341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:08.753341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.753341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:08.753341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:08.753341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.753341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:08.753341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.753341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:08.754488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.754488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:08.754488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:08.754488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:08.754488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:08.754488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:08.754488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:08.754488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:43:08.754488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:08.754488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:08.754488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.754488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:08.754929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:08.755373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.755373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:08.755373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.755373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:08.756691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:08.757168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:08.757752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.757752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:08.757752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:08.757752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.757752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:08.757752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.757752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:08.757959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:08.758118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:08.758193 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:08.758354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:08.758582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:08.758751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.759082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:08.759082 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.759082 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:08.759082 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:08.759082 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.759082 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:08.759082 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.759082 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:08.759649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:08.760081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:08.760081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:08.760081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.760081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:08.760081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:08.760081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.760081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:08.760081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.760081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:08.760364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.760364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:08.760364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:08.760364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:08.760364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:08.760364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:08.760364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:08.760364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:08.760364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:08.760364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.760364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:08.760743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:08.761134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:08.761614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:08.762158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:08.762158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:08.762158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:08.762158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:08.762158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.762158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:08.762158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:08.762158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.762158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:08.762158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.762158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:08.765832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:08.766221 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:08.766311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:08.766426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:08.766681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:08.766891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:08.767033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.767546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:08.767546 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.767546 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:08.767546 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:08.767546 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.767546 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:08.767546 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.767546 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:08.768342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.768342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:08.768342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:08.768342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:08.768342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:08.768342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:08.768342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:08.768342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.768342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:08.768342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:08.768342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:08.768342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.768739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:08.768739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:08.768739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:08.768739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:08.768739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:08.768739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:08.768739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:08.768739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:08.768739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.768739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:08.768739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:08.768739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.768739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:08.769092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.769092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:08.769362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:08.769877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:08.770453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:08.770453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:08.770453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:10.754773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:10.755969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:10.755969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:10.757600 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:10.796673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:10.796945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:10.797582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:10.797582 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:10.797582 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:10.797582 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:10.797582 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:10.797582 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:10.797582 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:10.797582 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:10.798533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:10.798533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:10.798533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:10.798533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:10.798533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:10.798533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:10.798533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:10.798533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:10.798533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:10.798533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:10.798533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:10.798533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:10.798996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:10.798996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:10.798996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:10.798996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:10.798996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:10.798996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:10.798996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:10.798996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:10.798996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:10.798996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:10.798996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:10.798996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:10.798996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:10.799458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:10.800000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:10.800808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:10.800808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:10.800808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:10.800808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:10.800808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:10.800808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:13.738919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:13.739786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:13.739786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:13.740983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:13.741358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:13.741707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.743164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:13.743164 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.743164 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:13.743164 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:13.743164 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.743164 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:13.743164 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.743164 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:13.743932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.743932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:13.743932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:13.743932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:13.743932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:13.743932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:13.743932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:13.743932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.743932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:13.743932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:13.743932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:13.743932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.744722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:13.744722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:13.744722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:13.744722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:13.744722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:13.744722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:13.744722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:13.744722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:13.744722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.744722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:13.744722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:13.744722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.744722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:13.746730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:13.747536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.748366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:13.748366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:13.748366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.748366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:13.748366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.748366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:13.748796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:13.749003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:13.749105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:13.749237 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:43:13.749806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:13.750078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:13.750786 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.750786 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:13.750786 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:13.750786 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.750786 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:13.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.750786 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:13.751516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:13.751997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:13.751997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:13.751997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.751997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:13.751997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:13.751997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.751997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:13.751997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.751997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:13.752320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.752320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:13.752320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:13.752320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:13.752320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:13.752320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:13.752320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:13.752320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:43:13.752320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:13.752320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:13.752320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.752320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:13.752682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:13.753083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.753083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:13.753083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.753083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:13.753294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:13.753804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:13.754331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.754331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:13.754331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:13.754331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.754331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:13.754331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.754331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:13.758154 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:13.758363 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:13.758471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:13.758589 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:13.759366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:13.759673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.760203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:13.760203 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.760203 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:13.760203 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:13.760203 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.760203 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:13.760203 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.760203 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:13.760921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:13.761375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:13.761375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:13.761375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.761375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:13.761375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:13.761375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.761375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:13.761375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.761375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:13.761797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.761797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:13.761797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:13.761797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:13.761797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:13.761797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:13.761797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:13.761797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:13.761797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:13.761797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.761797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:13.762148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:13.762564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.762564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:13.762564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.762564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:13.762823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:13.763342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:13.763880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.763880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:13.763880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.763880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:13.766342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:13.766622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:13.766806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:13.767023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:13.767380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:13.767595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:13.767742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.768328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:13.768328 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.768328 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:13.768328 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:13.768328 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.768328 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:13.768328 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.768328 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:13.769519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.769519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:13.769519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:13.769519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:13.769519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:13.769519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:13.769519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:13.769519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.769519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:13.769519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:13.769519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:13.769519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:13.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:13.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:13.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:13.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:13.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:13.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:13.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:13.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:13.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:13.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.769896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:13.770279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.770279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:13.770868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:13.771447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:13.771977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:13.771977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:13.771977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:15.755474 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:15.756090 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:15.756090 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:15.756443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:15.799673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:15.799853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:15.800410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:15.800410 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:15.800410 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:15.800410 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:15.800410 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:15.800410 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:15.800410 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:15.800410 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:15.802067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:15.802067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:15.802067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:15.802067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:15.802067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:15.802067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:15.802067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:15.802067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:15.802067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:15.802067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:15.802067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:15.802067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:15.802777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:15.802777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:15.802777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:15.802777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:15.802777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:15.802777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:15.802777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:15.802777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:15.802777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:15.802777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:15.802777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:15.802777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:15.802777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:15.803403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:15.803403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:15.804631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:15.805456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:15.806397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:15.806397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:15.806397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:18.739664 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:18.740457 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:18.740457 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:18.741113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:18.741350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:18.741534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:18.742345 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.742345 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:18.742345 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:18.742345 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.742345 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:18.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.742345 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:18.743265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.743265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:18.743265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:18.743265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:18.743265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:18.743265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:18.743265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:18.743265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.743265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:18.743265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:18.743265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:18.743265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.743685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:18.743685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:18.743685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:18.743685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:18.743685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:18.743685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:18.743685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:18.743685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:18.743685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.743685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:18.743685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:18.743685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.743685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:18.744074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.744074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:18.744447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:18.744906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.745441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:18.745441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.745441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:18.749285 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:18.749611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:18.749733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:18.749864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:43:18.750309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:18.750521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.751074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:18.751074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.751074 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:18.751074 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:18.751074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.751074 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:18.751074 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.751074 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:18.751820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:18.752352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:18.752352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:18.752352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.752352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:18.752352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:18.752352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.752352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:18.752352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.752352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:18.752702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.752702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:18.752702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:18.752702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:18.752702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:18.752702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:18.752702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:18.752702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:43:18.752702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:18.752702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:18.752702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.752702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:18.753093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:18.753490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.753490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:18.753490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.753490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:18.753734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:18.754248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:18.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:18.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:18.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:18.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.754766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:18.758635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:18.759023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:18.759149 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:18.759308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:18.759791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:18.760156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.760910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:18.760910 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.760910 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:18.760910 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:18.760910 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.760910 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:18.760910 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.760910 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:18.761741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:18.762112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:18.762112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:18.762112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.762112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:18.762112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:18.762112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.762112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:18.762112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.762112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:18.762699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.762699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:18.762699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:18.762699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:18.762699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:18.762699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:18.762699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:18.762699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:18.762699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:18.762699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.762699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:18.763024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:18.763396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.763396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:18.763396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.763396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:18.763822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:18.764353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:18.764911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.764911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:18.764911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.764911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:18.766841 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:18.767132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:18.767266 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:18.767430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:18.767857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:18.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:18.768446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.769108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:18.769108 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.769108 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:18.769108 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:18.769108 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.769108 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:18.769108 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.769108 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:18.770008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.770008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:18.770008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:18.770008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:18.770008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:18.770008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:18.770008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:18.770008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.770008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:18.770008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:18.770008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:18.770008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.770406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:18.770406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:18.770406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:18.770406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:18.770406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:18.770406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:18.770406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:18.770406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:18.770406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.770406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:18.770406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:18.770406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.770406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:18.770735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.770735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:18.771039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:18.771476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:18.771935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:18.771935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:18.771935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:20.756145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:20.756932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:20.756932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:20.758317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:20.788952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:20.789385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:20.790185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:20.790185 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:20.790185 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:20.790185 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:20.790185 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:20.790185 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:20.790185 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:20.790185 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:20.791030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:20.791030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:20.791030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:20.791030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:20.791030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:20.791030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:20.791030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:20.791030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:20.791030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:20.791030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:20.791030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:20.791030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:20.791822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:20.791822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:20.791822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:20.791822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:20.791822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:20.791822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:20.791822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:20.791822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:20.791822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:20.791822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:20.791822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:20.791822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:20.791822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:20.792569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:20.793450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:20.795264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:20.795264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:20.795264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:20.795264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:20.795264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:20.795264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:23.740557 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:23.741543 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:23.741543 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:23.742368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:23.742806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:23.743126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.744325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:23.744325 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.744325 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:23.744325 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:23.744325 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.744325 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:23.744325 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.744325 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:23.745627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.745627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:23.745627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:23.745627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:23.745627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:23.745627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:23.745627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:23.745627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.745627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:23.745627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:23.745627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:23.745627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.746410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:23.746410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:23.746410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:23.746410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:23.746410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:23.746410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:23.746410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:23.746410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:23.746410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.746410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:23.746410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:23.746410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.746410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:23.747116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:23.748683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.750260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:23.750260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:23.750260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.750260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:23.750260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.750260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:23.750260 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:23.750483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:23.750483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:23.750555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:23.753282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.753708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:23.754185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.754185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:23.754185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.754185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:23.754185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:23.754185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:23.754185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:23.754185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:23.754185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:23.754185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:43:23.754185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:23.754185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:23.754553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:23.754936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.755374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.755888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:23.761261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:23.761845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:23.761961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:23.762107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:23.762630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:23.763021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.763767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:23.763767 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.763767 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:23.763767 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:23.763767 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.763767 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:23.763767 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.763767 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:23.765112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:23.765587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:23.765976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:23.765976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:23.765976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:23.765976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:23.765976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.765976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:23.765976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:23.765976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:23.765976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.765976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:23.765976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:23.765976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:23.765976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:23.767455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:23.767455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:23.767455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:23.767455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:23.767455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.767455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:23.767455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:23.767455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.767455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:23.767455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.767455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:23.767455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:23.767722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.768109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:23.768526 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:23.768526 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:23.768526 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:23.768526 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:23.768673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:23.769043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:23.769300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.770085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:23.770085 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.770085 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:23.770085 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:23.770085 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.770085 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:23.770085 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.770085 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:23.771397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.771397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:23.771397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:23.771397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:23.771397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:23.771397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:23.771397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:23.771397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.771397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:23.771397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:23.771397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:23.771397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.771655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:23.771655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:23.771655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:23.771655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:23.771655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:23.771655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:23.771655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:23.771655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:23.771655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.771655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:23.771655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:23.771655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.771655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:23.771911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.771911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:23.773338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:23.773699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:23.774072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:23.774072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:23.774072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:25.756836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:25.757380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:25.757571 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:25.758852 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:25.794352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:25.794639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:25.795404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:25.795404 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:25.795404 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:25.795404 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:25.795404 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:25.795404 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:25.795404 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:25.795404 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:25.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:25.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:25.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:25.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:25.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:25.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:25.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:25.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:25.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:25.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:25.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:25.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:25.796831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:25.796831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:25.796831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:25.796831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:25.796831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:25.796831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:25.796831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:25.796831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:25.796831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:25.796831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:25.796831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:25.796831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:25.796831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:25.797228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:25.797228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:25.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:25.798004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:28.741245 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:28.742423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:28.742552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:28.743873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:28.744292 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:28.744560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.745572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:28.745572 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.745572 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:28.745572 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:28.745572 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.745572 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:28.745572 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.745572 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:28.746857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.746857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:28.746857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:28.746857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:28.746857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:28.746857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:28.746857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:28.746857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.746857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:28.746857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:28.746857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:28.746857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.747544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:28.747544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:28.747544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:28.747544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:28.747544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:28.747544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:28.747544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:28.747544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:28.747544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.747544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:28.747544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:28.747544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.747544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:28.748186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.748186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:28.748531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:28.749150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.749898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:28.749898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.749898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:28.750545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:28.750862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:28.751044 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:28.751245 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:43:28.751595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:28.751779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.752566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:28.752566 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.752566 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:28.752566 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:28.752566 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.752566 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:28.752566 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.752566 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:28.753565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:28.754396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:28.754396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:28.754396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.754396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:28.754396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:28.754396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.754396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:28.754396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.754396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:28.754859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.754859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:28.754859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:28.754859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:28.754859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:28.754859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:28.754859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:28.754859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:43:28.754859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:28.754859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:28.754859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.754859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:28.755514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:28.756147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.756147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:28.756147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.756147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:28.756367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:28.756863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:28.757648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.757648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:28.757648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:28.757648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.757648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:28.757648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.757648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:28.761845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:28.762343 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:28.762496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:28.762685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:28.763308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:28.763724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.764965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:28.764965 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.764965 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:28.764965 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:28.764965 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.764965 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:28.764965 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.764965 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:28.765908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:28.766767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:28.766767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:28.766767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.766767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:28.766767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:28.766767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.766767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:28.766767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.766767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:28.768537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.768537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:28.768537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:28.768537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:28.768537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:28.768537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:28.768537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:28.768537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:28.768537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:28.768537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.768537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:28.769331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:28.769894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:28.771184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:28.772261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:28.772261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:28.772261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:28.772261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:28.772261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.772261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:28.772261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:28.772261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.772261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:28.772261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.772261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:28.772261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:28.772261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:28.772745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:28.772745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:28.772745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:28.772745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:28.772745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.773103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:28.773103 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.773103 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:28.773103 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:28.773103 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.773103 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:28.773103 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.773103 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:28.774342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.774342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:28.774342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:28.774342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:28.774342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:28.774342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:28.774342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:28.774342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.774342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:28.774342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:28.774342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:28.774342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.774705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:28.774705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:28.774705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:28.774705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:28.774705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:28.774705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:28.774705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:28.774705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:28.774705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.774705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:28.774705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:28.774705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.774705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:28.775085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.775085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:28.775430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:28.775865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:28.776372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:28.776372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:28.776372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:30.757900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:30.759290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:30.759507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:30.760905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:30.803099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:30.803601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:30.804764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:30.804764 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:30.804764 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:30.804764 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:30.804764 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:30.804764 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:30.804764 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:30.804764 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:30.805900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:30.805900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:30.805900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:30.805900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:30.805900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:30.805900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:30.805900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:30.805900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:30.805900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:30.805900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:30.805900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:30.805900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:30.806640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:30.806640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:30.806640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:30.806640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:30.806640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:30.806640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:30.806640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:30.806640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:30.806640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:30.806640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:30.806640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:30.806640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:30.806640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:30.807235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:30.807644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:30.808123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:30.808123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:30.808123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:30.808123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:30.808123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:30.808123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:33.742270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:33.743202 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:33.743202 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:33.744388 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:33.744759 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:33.745056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.745940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:33.745940 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.745940 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:33.745940 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:33.745940 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.745940 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:33.745940 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.745940 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:33.747529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.747529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:33.747529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:33.747529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:33.747529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:33.747529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:33.747529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:33.747529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.747529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:33.747529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:33.747529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:33.747529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.748179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:33.748179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:33.748179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:33.748179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:33.748179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:33.748179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:33.748179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:33.748179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:33.748179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.748179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:33.748179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:33.748179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.748179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:33.748831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:33.749762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.750793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:33.750793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:33.750793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.750793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:33.750793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.750793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:33.751163 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:33.751639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:33.751731 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:33.751868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:43:33.752535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:33.752766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.753605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:33.753605 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.753605 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:33.753605 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:33.753605 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.753605 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:33.753605 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.753605 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:33.754368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:33.754805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:33.754805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:33.754805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.754805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:33.754805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:33.754805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.754805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:33.754805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.754805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:33.755163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.755163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:33.755163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:33.755163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:33.755163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:33.755163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:33.755163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:33.755163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:43:33.755163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:33.755163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:33.755163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.755163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:33.755515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:33.755843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.755843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:33.755843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.755843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:33.756162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:33.756596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:33.757062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.757062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:33.757062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:33.757062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.757062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:33.757062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.757062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:33.762830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:33.763923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:33.764016 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:33.764124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:33.764861 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:33.765503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.766346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:33.766346 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.766346 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:33.766346 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:33.766346 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.766346 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:33.766346 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.766346 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:33.766920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:33.767361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:33.767361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:33.767361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.767361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:33.767361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:33.767361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.767361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:33.767361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.767361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:33.767595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.767595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:33.767595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:33.767595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:33.767595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:33.767595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:33.767595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:33.767595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:33.767595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:33.767595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.767595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:33.767922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:33.768308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:33.768701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:33.769226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:33.769226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:33.769226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:33.769226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:33.769226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.769226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:33.769226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:33.769226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.769226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:33.769226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.769226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:33.770732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:33.770960 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:33.771051 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:33.771166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:33.771882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:33.772301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:33.772466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.773036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:33.773036 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.773036 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:33.773036 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:33.773036 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.773036 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:33.773036 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.773036 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:33.773641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.773641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:33.773641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:33.773641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:33.773641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:33.773641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:33.773641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:33.773641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.773641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:33.773641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:33.773641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:33.773641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.774023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:33.774023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:33.774023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:33.774023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:33.774023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:33.774023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:33.774023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:33.774023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:33.774023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.774023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:33.774023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:33.774023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.774023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:33.774383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.774383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:33.774444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:33.774889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:33.775481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:33.775481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:33.775481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:35.759085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:35.759893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:35.759893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:35.761083 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:35.794894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:35.795155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:35.795840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:35.795840 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:35.795840 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:35.795840 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:35.795840 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:35.795840 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:35.795840 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:35.795840 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:35.796604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:35.796604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:35.796604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:35.796604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:35.796604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:35.796604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:35.796604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:35.796604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:35.796604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:35.796604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:35.796604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:35.796604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:35.797416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:35.797416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:35.797416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:35.797416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:35.797416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:35.797416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:35.797416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:35.797416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:35.797416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:35.797416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:35.797416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:35.797416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:35.797416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:35.798236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:35.799189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:35.800497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:35.800497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:35.800497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:35.800497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:35.800497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:35.800497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:38.743159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:38.744002 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:38.744068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:38.747346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:38.747346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:38.747346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.747346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:38.747346 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.747346 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:38.747346 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:38.747346 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.747346 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:38.747346 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.747346 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:38.748462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.748462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:38.748462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:38.748462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:38.748462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:38.748462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:38.748462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:38.748462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.748462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:38.748462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:38.748462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:38.748462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.748832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:38.748832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:38.748832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:38.748832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:38.748832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:38.748832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:38.748832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:38.748832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:38.748832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.748832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:38.748832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:38.748832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.748832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:38.749165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.749165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:38.750088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:38.750510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.750975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:38.750975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.750975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:38.751792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:38.752053 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:38.752137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:38.752239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:43:38.752810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:38.753035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.753674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:38.753674 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.753674 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:38.753674 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:38.753674 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.753674 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:38.753674 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.753674 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:38.754349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:38.754740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:38.754740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:38.754740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.754740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:38.754740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:38.754740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.754740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:38.754740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.754740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:38.754967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.754967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:38.754967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:38.754967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:38.754967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:38.754967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:38.754967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:38.754967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:43:38.754967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:38.754967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:38.754967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.754967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:38.755326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:38.755667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:38.756015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:38.756497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:38.756497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:38.756497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:38.756497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:38.756497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:38.756497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.756497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:38.756497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:38.756497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.756497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:38.756497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.756497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:38.763531 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:38.763860 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:38.764043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:38.764271 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:38.765117 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:38.765341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.766021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:38.766021 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.766021 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:38.766021 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:38.766021 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.766021 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:38.766021 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.766021 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:38.766854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:38.767587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:38.767587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:38.767587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.767587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:38.767587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:38.767587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.767587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:38.767587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.767587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:38.768037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.768037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:38.768037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:38.768037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:38.768037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:38.768037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:38.768037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:38.768037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:38.768037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:38.768037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.768037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:38.768561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:38.769138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.769138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:38.769138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.769138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:38.769353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:38.770429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:38.770991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.770991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:38.770991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.770991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:38.771118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:38.771367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:38.771488 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:38.771663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:38.772265 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:38.772536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:38.772698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.773330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:38.773330 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.773330 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:38.773330 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:38.773330 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.773330 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:38.773330 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.773330 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:38.774482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.774482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:38.774482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:38.774482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:38.774482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:38.774482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:38.774482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:38.774482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.774482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:38.774482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:38.774482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:38.774482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.774865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:38.774865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:38.774865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:38.774865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:38.774865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:38.774865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:38.774865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:38.774865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:38.774865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.774865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:38.774865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:38.774865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.774865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:38.775241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.775241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:38.775854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:38.776353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:38.776901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:38.776901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:38.776901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:40.759744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:40.762685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:40.762792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:40.763682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:40.802864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:40.803158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:40.803836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:40.803836 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:40.803836 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:40.803836 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:40.803836 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:40.803836 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:40.803836 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:40.803836 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:40.804618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:40.804618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:40.804618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:40.804618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:40.804618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:40.804618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:40.804618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:40.804618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:40.804618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:40.804618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:40.804618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:40.804618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:40.805255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:40.805255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:40.805255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:40.805255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:40.805255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:40.805255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:40.805255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:40.805255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:40.805255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:40.805255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:40.805255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:40.805255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:40.805255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:40.805846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:40.806483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:40.807537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:40.807537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:40.807537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:40.807537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:40.807537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:40.807537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:43.743792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:43.745077 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:43.745077 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:43.747880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:43.747880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:43.747880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.748193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:43.748193 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.748193 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:43.748193 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:43.748193 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.748193 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:43.748193 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.748193 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:43.749145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.749145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:43.749145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:43.749145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:43.749145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:43.749145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:43.749145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:43.749145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.749145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:43.749145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:43.749145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:43.749145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.752315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:43.752315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:43.752315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:43.752315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:43.752315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:43.752315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:43.752315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:43.752315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:43.752315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.752315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:43.752315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:43.752315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.752315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:43.755750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:43.756913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.757456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:43.757456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:43.757456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.757456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:43.757456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.757456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:43.757456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:43.757456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:43.757456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:43.757456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:43.757806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.758179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:43.758724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.758724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:43.758724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.758724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:43.758724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:43.758724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:43.758724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:43.758724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:43.758724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:43.758724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:43:43.758724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:43.758724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:43.759078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:43.759572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.759956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.760452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:43.763944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:43.764360 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:43.764516 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:43.764676 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:43.765875 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:43.765875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.766040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:43.766040 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.766040 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:43.766040 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:43.766040 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.766040 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:43.766040 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.766040 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:43.766869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:43.767275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:43.767275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:43.767275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.767275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:43.767275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:43.767275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.767275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:43.767275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.767275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:43.767799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.767799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:43.767799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:43.767799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:43.767799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:43.767799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:43.767799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:43.767799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:43.767799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:43.767799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.767799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:43.768189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:43.768632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.768632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:43.768632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.768632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:43.768908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:43.769466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:43.770114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.770114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:43.770114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.770114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:43.771474 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:43.771796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:43.771946 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:43.772141 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:43.772573 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:43.772894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:43.773118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.773993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:43.773993 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.773993 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:43.773993 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:43.773993 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.773993 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:43.773993 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.773993 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:43.774846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.774846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:43.774846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:43.774846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:43.774846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:43.774846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:43.774846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:43.774846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.774846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:43.774846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:43.774846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:43.774846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.775295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:43.775295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:43.775295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:43.775295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:43.775295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:43.775295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:43.775295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:43.775295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:43.775295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.775295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:43.775295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:43.775295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.775295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:43.775751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.775751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:43.775845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:43.776325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:43.776848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:43.776848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:43.776848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:45.762923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:45.764019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:45.764019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:45.764134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:45.798729 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:45.799000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:45.799685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:45.799685 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:45.799685 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:45.799685 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:45.799685 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:45.799685 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:45.799685 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:45.799685 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:45.800429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:45.800429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:45.800429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:45.800429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:45.800429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:45.800429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:45.800429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:45.800429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:45.800429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:45.800429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:45.800429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:45.800429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:45.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:45.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:45.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:45.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:45.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:45.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:45.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:45.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:45.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:45.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:45.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:45.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:45.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:45.801301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:45.801833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:45.802615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:45.802615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:45.802615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:45.802615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:45.802615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:45.802615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:48.744715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:48.745567 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:48.745567 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:48.746511 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:48.746703 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:48.746847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.747304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:48.747304 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.747304 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:48.747304 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:48.747304 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.747304 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:48.747304 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.747304 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:48.747986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.747986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:48.747986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:48.747986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:48.747986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:48.747986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:48.747986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:48.747986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.747986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:48.747986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:48.747986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:48.747986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.748514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:48.748514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:48.748514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:48.748514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:48.748514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:48.748514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:48.748514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:48.748514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:48.748514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.748514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:48.748514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:48.748514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.748514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:48.749024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:48.749712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.750552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:48.750552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:48.750552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.750552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:48.750552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.750552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:48.753270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:48.753823 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:48.754068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:48.754360 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:43:48.754972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:48.755135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.755632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:48.755632 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.755632 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:48.755632 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:48.755632 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.755632 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:48.755632 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.755632 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:48.756708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:48.757138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:48.757138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:48.757138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.757138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:48.757138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:48.757138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.757138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:48.757138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.757138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:48.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:48.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:48.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:48.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:48.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:48.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:48.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:43:48.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:48.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:48.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:48.758342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:48.758751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.758751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:48.758751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.758751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:48.759528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:48.760061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:48.760550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.760550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:48.760550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:48.760550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.760550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:48.760550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.760550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:48.764319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:48.764727 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:48.764851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:48.765195 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:48.765592 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:48.765802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.766475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:48.766475 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.766475 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:48.766475 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:48.766475 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.766475 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:48.766475 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.766475 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:48.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:48.767602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:48.767602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:48.767602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.767602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:48.767602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:48.767602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.767602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:48.767602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.767602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:48.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:48.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:48.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:48.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:48.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:48.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:48.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:48.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:48.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:48.768518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:48.768917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.768917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:48.768917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.768917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:48.773846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:48.774413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:48.775503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.775503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:48.775503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.775503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:48.775503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:48.775503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:48.775503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:48.775503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:48.775938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:48.776139 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:48.776319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.776900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:48.776900 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.776900 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:48.776900 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:48.776900 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.776900 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:48.776900 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.776900 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:48.777683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.777683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:48.777683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:48.777683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:48.777683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:48.777683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:48.777683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:48.777683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.777683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:48.777683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:48.777683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:48.777683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.778068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:48.778068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:48.778068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:48.778068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:48.778068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:48.778068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:48.778068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:48.778068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:48.778068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.778068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:48.778068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:48.778068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.778068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:48.778488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.778488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:48.778612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:48.779121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:48.779614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:48.779614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:48.779614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:50.763428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:50.763929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:50.763995 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:50.764839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:50.802469 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:50.802840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:50.803589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:50.803589 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:50.803589 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:50.803589 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:50.803589 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:50.803589 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:50.803589 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:50.803589 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:50.804875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:50.804875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:50.804875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:50.804875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:50.804875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:50.804875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:50.804875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:50.804875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:50.804875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:50.804875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:50.804875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:50.804875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:50.805198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:50.805198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:50.805198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:50.805198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:50.805198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:50.805198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:50.805198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:50.805198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:50.805198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:50.805198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:50.805198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:50.805198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:50.805198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:50.805486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:50.805486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:50.806503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:50.806856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:50.807297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:50.807297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:50.807297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:53.745444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:53.746130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:53.746130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:53.747272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:53.747446 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:53.747562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.748073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:53.748073 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.748073 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:53.748073 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:53.748073 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.748073 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:53.748073 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.748073 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:53.748824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.748824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:53.748824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:53.748824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:53.748824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:53.748824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:53.748824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:53.748824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.748824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:53.748824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:53.748824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:53.748824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.749578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:53.749578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:53.749578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:53.749578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:53.749578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:53.749578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:53.749578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:53.749578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:53.749578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.749578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:53.749578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:53.749578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.749578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:53.750181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:53.751086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.752045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:53.752045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:53.752045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.752045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:53.752045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.752045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:53.754173 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:53.754173 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:53.754173 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:53.754375 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:43:53.754833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:53.755092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:53.755526 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.755526 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:53.755526 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:53.755526 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.755526 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:53.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.755526 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:53.756059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:53.756763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:53.757418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:53.757418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:53.757418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:43:53.757418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:53.757418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:53.757418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.757418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:53.757418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:53.757418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:53.757418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.757418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:53.757418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:53.757418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:53.757807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:53.757807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:53.757807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:53.757807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:53.757807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:53.757807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.757807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:53.757807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:53.757807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.757807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:53.757807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.757807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:53.758136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:53.758579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:53.759154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.759154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:53.759154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:53.759154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.759154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:53.759154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.759154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:53.764785 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:53.765223 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:53.765472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:53.765729 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:53.766167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:53.766409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.766980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:53.766980 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.766980 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:53.766980 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:53.766980 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.766980 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:53.766980 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.766980 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:53.768741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:53.769251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:53.769251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:53.769251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.769251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:53.769251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:53.769251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.769251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:53.769251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.769251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:53.770361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.770361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:53.770361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:53.770361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:53.770361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:53.770361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:53.770361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:53.770361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:53.770361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:53.770361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.770361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:53.770759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:53.771235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.771235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:53.771235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.771235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:53.772672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:53.773491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:53.774006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.774006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:53.774006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.774006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:53.774805 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:53.775161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:53.775246 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:53.775409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:53.775848 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:53.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:53.776244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.776834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:53.776834 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.776834 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:53.776834 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:53.776834 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.776834 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:53.776834 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.776834 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:53.777805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.777805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:53.777805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:53.777805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:53.777805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:53.777805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:53.777805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:53.777805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.777805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:53.777805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:53.777805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:53.777805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.778271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:53.778271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:53.778271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:53.778271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:53.778271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:53.778271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:53.778271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:53.778271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:53.778271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.778271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:53.778271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:53.778271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.778271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:53.778675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.778675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:53.778821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:53.779609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:53.780116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:53.780116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:53.780116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:55.764038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:55.764880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:55.764880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:55.766539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:55.801595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:55.801824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:55.802437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:55.802437 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:55.802437 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:55.802437 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:55.802437 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:55.802437 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:55.802437 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:55.802437 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:55.803264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:55.803264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:55.803264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:55.803264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:55.803264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:55.803264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:55.803264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:55.803264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:55.803264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:55.803264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:55.803264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:55.803264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:55.803882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:55.803882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:55.803882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:55.803882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:55.803882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:55.803882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:55.803882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:55.803882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:55.803882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:55.803882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:55.803882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:55.803882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:55.803882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:55.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:55.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:55.805435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:55.805435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:55.805435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:55.805435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:55.805435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:55.805435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:58.745975 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:58.746695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:58.746695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:58.747420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:58.747662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:58.747811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.748497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:58.748497 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.748497 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:58.748497 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:58.748497 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.748497 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:58.748497 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.748497 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:58.749637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.749637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:58.749637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:58.749637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:58.749637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:58.749637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:58.749637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:58.749637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.749637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:58.749637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:58.749637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:58.749637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.750279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:58.750279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:58.750279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:58.750279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:58.750279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:58.750279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:58.750279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:58.750279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:58.750279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.750279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:58.750279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:58.750279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.750279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:58.751036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:58.751635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.752357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:58.752357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:58.752357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.752357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:58.752357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.752357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:58.754238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:58.754586 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:58.754729 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:58.754889 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:43:58.755553 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:58.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.756699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:58.756699 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.756699 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:58.756699 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:58.756699 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.756699 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:58.756699 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.756699 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:58.757727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:58.758360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:58.758360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:58.758360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.758360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:58.758360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:58.758360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.758360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:58.758360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.758360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:58.758874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.758874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:58.758874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:58.758874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:58.758874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:58.758874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:58.758874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:58.758874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:43:58.758874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:58.758874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:58.758874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.758874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:58.759390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:58.760004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.760004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:58.760004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.760004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:58.760497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:58.761357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:58.761929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.761929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:58.761929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:58.761929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.761929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:58.761929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.761929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:58.765336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:58.765767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:58.765856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:58.765998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:58.766639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:58.766928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.767542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:58.767542 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.767542 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:58.767542 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:58.767542 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.767542 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:58.767542 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.767542 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:43:58.768371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:43:58.768768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:58.768768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:43:58.768768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.768768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:43:58.768768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:43:58.768768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.768768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:43:58.768768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.768768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:43:58.769356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.769356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:58.769356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:58.769356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:58.769356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:58.769356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:58.769356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:58.769356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:58.769356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:58.769356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.769356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:58.769654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:58.770118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.770118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:58.770118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.770118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:58.771173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:58.771560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:58.772060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.772060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:58.772060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.772060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:43:58.775338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:43:58.775666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:43:58.775780 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:43:58.776039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:43:58.776781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:58.777062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:43:58.777217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.777840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:43:58.777840 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.777840 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:43:58.777840 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:43:58.777840 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.777840 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:43:58.777840 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.777840 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:43:58.778811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.778811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:43:58.778811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:58.778811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:43:58.778811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:43:58.778811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:58.778811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:58.778811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.778811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:58.778811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:58.778811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:43:58.778811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.779077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:43:58.779077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:58.779077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:43:58.779077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:58.779077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:43:58.779077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:43:58.779077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:58.779077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:43:58.779077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.779077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:43:58.779077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:43:58.779077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.779077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:43:58.779340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.779340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:43:58.779887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:43:58.780326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:43:58.780794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:43:58.780794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:43:58.780794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:00.764814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:00.765385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:00.765385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:00.765995 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:00.793956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:00.794215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:00.794813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:00.794813 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:00.794813 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:00.794813 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:00.794813 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:00.794813 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:00.794813 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:00.794813 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:00.795762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:00.795762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:00.795762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:00.795762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:00.795762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:00.795762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:00.795762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:00.795762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:00.795762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:00.795762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:00.795762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:00.795762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:00.796145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:00.796145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:00.796145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:00.796145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:00.796145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:00.796145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:00.796145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:00.796145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:00.796145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:00.796145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:00.796145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:00.796145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:00.796145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:00.796536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:00.796536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:00.796935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:00.798062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:00.798630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:00.798630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:00.798630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:03.746483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:03.747021 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:03.747066 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:03.750242 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:03.750242 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:03.750242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.750242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:03.750242 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.750242 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:03.750242 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:03.750242 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.750242 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:03.750242 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.750242 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:03.750242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:03.750664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:03.750961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:03.750961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:03.750961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:03.750961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:03.750961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:03.750961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.750961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:03.750961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:03.750961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.750961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:03.750961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.750961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:03.751259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:03.751599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.751987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:03.754681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:03.754992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:03.755111 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:03.755256 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:44:03.755605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:03.755775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.756228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:03.756228 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.756228 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:03.756228 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:03.756228 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.756228 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:03.756228 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.756228 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:03.756988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:03.757529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:03.757529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:03.757529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.757529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:03.757529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:03.757529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.757529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:03.757529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.757529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:03.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:03.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:03.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:03.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:03.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:03.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:03.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:44:03.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:03.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:03.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.757940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:03.758489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:03.758898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.758898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:03.758898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.758898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:03.759018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:03.759486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:03.759993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.759993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:03.759993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:03.759993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.759993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:03.759993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.759993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:03.765919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:03.767329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:03.767329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:03.767329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:03.768887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:03.768887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.769050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:03.769050 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.769050 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:03.769050 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:03.769050 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.769050 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:03.769050 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.769050 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:03.769794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:03.770086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:03.770086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:03.770086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.770086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:03.770086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:03.770086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.770086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:03.770086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.770086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:03.770704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.770704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:03.770704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:03.770704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:03.770704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:03.770704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:03.770704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:03.770704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:03.770704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:03.770704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.770704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:03.770951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:03.771200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.771200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:03.771200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.771200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:03.771812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:03.772148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:03.772516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.772516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:03.772516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.772516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:03.775777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:03.779613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:03.779685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:03.779827 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:03.780217 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:03.780539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:03.780652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.781356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:03.781356 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.781356 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:03.781356 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:03.781356 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.781356 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:03.781356 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.781356 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:03.781981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.781981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:03.781981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:03.781981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:03.781981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:03.781981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:03.781981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:03.781981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.781981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:03.781981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:03.781981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:03.781981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.782358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:03.782358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:03.782358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:03.782358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:03.782358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:03.782358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:03.782358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:03.782358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:03.782358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.782358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:03.782358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:03.782358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.782358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:03.782637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.782637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:03.784715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:03.785085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:03.785549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:03.785549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:03.785549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:05.765566 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:05.766164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:05.766359 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:05.767840 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:05.769851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:44:05.800638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:05.801040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:05.801889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:05.801889 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:05.801889 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:05.801889 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:05.801889 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:05.801889 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:05.801889 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:05.801889 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:05.803628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:05.803628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:05.803628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:05.803628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:05.803628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:05.803628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:05.803628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:05.803628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:05.803628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:05.803628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:05.803628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:05.803628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:05.804082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:05.804082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:05.804082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:05.804082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:05.804082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:05.804082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:05.804082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:05.804082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:05.804082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:05.804082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:05.804082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:05.804082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:05.804082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:05.804521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:05.805022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:05.805727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:05.805727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:05.805727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:05.805727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:05.805727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:05.805727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:08.747314 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:08.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:08.748088 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:08.748796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:08.749120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:08.749424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.750283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:08.750283 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.750283 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:08.750283 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:08.750283 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.750283 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:08.750283 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.750283 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:08.751676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.751676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:08.751676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:08.751676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:08.751676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:08.751676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:08.751676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:08.751676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.751676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:08.751676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:08.751676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:08.751676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.752230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:08.752230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:08.752230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:08.752230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:08.752230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:08.752230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:08.752230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:08.752230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:08.752230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.752230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:08.752230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:08.752230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.752230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:08.752676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.752676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:08.753747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:08.754493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.755087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:08.755087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.755087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:08.755317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:08.755697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:08.755866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:08.756047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:44:08.756431 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:08.756637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.758105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:08.758105 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.758105 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:08.758105 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:08.758105 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.758105 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:08.758105 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.758105 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:08.759798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:08.760328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:08.760328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:08.760328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.760328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:08.760328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:08.760328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.760328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:08.760328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.760328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:08.761173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.761173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:08.761173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:08.761173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:08.761173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:08.761173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:08.761173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:08.761173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:44:08.761173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:08.761173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:08.761173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.761173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:08.761612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:08.761989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.761989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:08.761989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.761989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:08.762876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:08.763430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:08.763954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.763954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:08.763954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:08.763954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.763954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:08.763954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.763954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:08.767182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:08.767581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:08.767691 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:08.767932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:08.768322 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:08.768560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.769112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:08.769112 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.769112 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:08.769112 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:08.769112 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.769112 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:08.769112 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.769112 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:08.769968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:08.770553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:08.770553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:08.770553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.770553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:08.770553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:08.770553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.770553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:08.770553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.770553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:08.770977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.770977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:08.770977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:08.770977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:08.770977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:08.770977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:08.770977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:08.770977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:08.770977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:08.770977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.770977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:08.771509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:08.772305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.772305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:08.772305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.772305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:08.772490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:08.773072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:08.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:08.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:08.779870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:08.780270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:08.780433 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:08.780697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:08.781109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:08.781355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:08.781503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.782116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:08.782116 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.782116 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:08.782116 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:08.782116 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.782116 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:08.782116 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.782116 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:08.783650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.783650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:08.783650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:08.783650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:08.783650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:08.783650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:08.783650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:08.783650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.783650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:08.783650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:08.783650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:08.783650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.784171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:08.784171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:08.784171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:08.784171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:08.784171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:08.784171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:08.784171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:08.784171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:08.784171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.784171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:08.784171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:08.784171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.784171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:08.784632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.784632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:08.785339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:08.785932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:08.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:08.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:08.786558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:10.766286 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:10.767043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:10.767043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:10.767753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:10.801526 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:10.801767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:10.802312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:10.802312 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:10.802312 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:10.802312 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:10.802312 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:10.802312 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:10.802312 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:10.802312 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:10.803188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:10.803188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:10.803188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:10.803188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:10.803188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:10.803188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:10.803188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:10.803188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:10.803188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:10.803188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:10.803188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:10.803188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:10.803994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:10.803994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:10.803994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:10.803994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:10.803994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:10.803994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:10.803994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:10.803994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:10.803994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:10.803994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:10.803994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:10.803994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:10.803994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:10.804589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:10.805422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:10.806234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:10.806234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:10.806234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:10.806234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:10.806234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:10.806234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:13.747843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:13.748887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:13.749016 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:13.749967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:13.750411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:13.750614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.751449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:13.751449 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.751449 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:13.751449 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:13.751449 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.751449 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:13.751449 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.751449 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:13.752464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.752464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:13.752464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:13.752464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:13.752464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:13.752464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:13.752464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:13.752464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.752464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:13.752464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:13.752464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:13.752464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.753022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:13.753022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:13.753022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:13.753022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:13.753022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:13.753022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:13.753022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:13.753022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:13.753022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.753022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:13.753022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:13.753022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.753022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:13.753596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:13.754247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.754972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:13.754972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:13.754972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.754972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:13.754972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.754972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:13.755910 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:13.756283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:13.756392 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:13.756538 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:44:13.756995 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:13.757317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.758001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:13.758001 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.758001 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:13.758001 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:13.758001 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.758001 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:13.758001 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.758001 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:13.758674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:13.759077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:13.759077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:13.759077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.759077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:13.759077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:13.759077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.759077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:13.759077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.759077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:13.759346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.759346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:13.759346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:13.759346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:13.759346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:13.759346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:13.759346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:13.759346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:44:13.759346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:13.759346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:13.759346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.759346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:13.759682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:13.760046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.760046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:13.760046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.760046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:13.760158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:13.760592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:13.761074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.761074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:13.761074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:13.761074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.761074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:13.761074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.761074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:13.767517 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:13.767907 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:13.768034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:13.768175 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:13.768546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:13.768870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.769609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:13.769609 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.769609 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:13.769609 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:13.769609 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.769609 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:13.769609 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.769609 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:13.770578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:13.770997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:13.770997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:13.770997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.770997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:13.770997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:13.770997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.770997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:13.770997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.770997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:13.771563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.771563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:13.771563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:13.771563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:13.771563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:13.771563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:13.771563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:13.771563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:13.771563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:13.771563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.771563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:13.771875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:13.772274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.772274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:13.772274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.772274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:13.772714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:13.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:13.773692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.773692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:13.773692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.773692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:13.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:13.780837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:13.780959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:13.781152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:13.781636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:13.782004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:13.782272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.782997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:13.782997 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.782997 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:13.782997 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:13.782997 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.782997 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:13.782997 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.782997 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:13.784189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.784189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:13.784189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:13.784189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:13.784189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:13.784189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:13.784189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:13.784189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.784189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:13.784189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:13.784189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:13.784189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.784568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:13.784568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:13.784568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:13.784568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:13.784568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:13.784568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:13.784568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:13.784568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:13.784568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.784568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:13.784568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:13.784568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.784568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:13.785034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.785034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:13.785278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:13.785727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:13.786245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:13.786245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:13.786245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:15.767788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:15.767788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:15.767788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:15.768341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:15.796897 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:15.797098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:15.797671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:15.797671 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:15.797671 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:15.797671 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:15.797671 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:15.797671 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:15.797671 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:15.797671 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:15.798422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:15.798422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:15.798422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:15.798422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:15.798422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:15.798422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:15.798422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:15.798422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:15.798422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:15.798422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:15.798422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:15.798422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:15.798715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:15.798715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:15.798715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:15.798715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:15.798715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:15.798715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:15.798715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:15.798715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:15.798715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:15.798715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:15.798715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:15.798715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:15.798715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:15.798958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:15.798958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:15.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:15.799463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:15.799796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:15.799796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:15.799796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:18.748914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:18.749961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:18.749961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:18.753739 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:18.753973 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:18.754134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.754848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:18.754848 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.754848 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:18.754848 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:18.754848 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.754848 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:18.754848 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.754848 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:18.756405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.756405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:18.756405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:18.756405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:18.756405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:18.756405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:18.756405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:18.756405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.756405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:18.756405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:18.756405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:18.756405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:18.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:18.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:18.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:18.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:18.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:18.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:18.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:18.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:18.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:18.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.756935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:18.757433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.757433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:18.758003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:18.758638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.759434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:18.759434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.759434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:18.759434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:18.759909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:18.760054 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:18.760251 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:44:18.760571 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:18.760736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.761196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:18.761196 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.761196 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:18.761196 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:18.761196 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.761196 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:18.761196 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.761196 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:18.762079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:18.762692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:18.762692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:18.762692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.762692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:18.762692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:18.762692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.762692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:18.762692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.762692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:18.763291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.763291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:18.763291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:18.763291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:18.763291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:18.763291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:18.763291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:18.763291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:44:18.763291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:18.763291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:18.763291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.763291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:18.763735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:18.764185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.764185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:18.764185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.764185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:18.764558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:18.765054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:18.765652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.765652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:18.765652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:18.765652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.765652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:18.765652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.765652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:18.767885 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:18.768169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:18.768397 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:18.768564 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:18.769083 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:18.769363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.769874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:18.769874 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.769874 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:18.769874 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:18.769874 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.769874 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:18.769874 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.769874 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:18.770404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:18.770879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:18.770879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:18.770879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.770879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:18.770879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:18.770879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.770879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:18.770879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.770879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:18.771103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.771103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:18.771103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:18.771103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:18.771103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:18.771103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:18.771103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:18.771103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:18.771103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:18.771103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.771103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:18.771387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:18.771672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.771672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:18.771672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.771672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:18.771758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:18.772162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:18.772532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.772532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:18.772532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.772532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:18.781089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:18.781287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:18.781405 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:18.781550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:18.782108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:18.782402 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:18.782561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.783193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:18.783193 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.783193 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:18.783193 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:18.783193 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.783193 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:18.783193 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.783193 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:18.784264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.784264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:18.784264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:18.784264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:18.784264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:18.784264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:18.784264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:18.784264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.784264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:18.784264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:18.784264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:18.784264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.784596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:18.784596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:18.784596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:18.784596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:18.784596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:18.784596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:18.784596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:18.784596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:18.784596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.784596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:18.784596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:18.784596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.784596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:18.784920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.784920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:18.785233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:18.785836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:18.786363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:18.786363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:18.786363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:20.767394 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:20.767880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:20.767880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:20.768373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:20.801277 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:20.801277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:20.801277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:20.801277 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:20.801277 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:20.801277 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:20.801277 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:20.801277 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:20.801277 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:20.801277 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:20.801997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:20.801997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:20.801997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:20.801997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:20.801997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:20.801997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:20.801997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:20.801997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:20.801997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:20.801997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:20.801997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:20.801997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:20.802736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:20.802736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:20.802736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:20.802736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:20.802736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:20.802736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:20.802736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:20.802736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:20.802736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:20.802736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:20.802736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:20.802736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:20.802736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:20.803136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:20.803613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:20.804220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:20.804220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:20.804220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:20.804220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:20.804220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:20.804220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:23.749931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:23.750581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:23.750581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:23.796269 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:23.796269 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:23.796269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.796269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:23.796269 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.796269 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:23.796269 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:23.796269 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.796269 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:23.796269 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.796269 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:23.796269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:23.797251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:23.797925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:23.797925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:23.797925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:23.797925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:23.797925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:23.797925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.797925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:23.797925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:23.797925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.797925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:23.797925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.797925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:23.798488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:23.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.800669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:23.800669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.800669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:23.800669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:23.800669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:23.800669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:23.800669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:44:23.800669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:23.801765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.802832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:23.803893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.803893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:23.803893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:23.803893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:23.803893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:23.803893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:23.803893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:23.803893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:44:23.803893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:23.803893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:23.803893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.803893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:23.804895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:23.805596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:23.806396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:23.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:23.807299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:23.807299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:23.807299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:23.807299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.807299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:23.807299 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.807299 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:23.807299 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:23.807299 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.807299 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:23.807299 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.807299 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:23.807539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:23.807892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:23.808171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:23.808171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:23.808171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:23.808171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:23.808171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.808171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:23.808171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:23.808171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:23.808171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.808171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:23.808171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:23.808171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:23.808171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:23.808491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:23.808491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:23.808491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:23.808491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:23.808491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.808491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:23.808491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:23.808491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.808491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:23.808491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.808491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:23.808491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:23.808888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.809401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:23.809818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:23.809818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:23.809818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:23.809818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:23.809818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:23.809818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:23.809968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:23.810424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:23.810799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.810799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:23.810799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:23.810799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.810799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:23.810799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.810799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:23.810799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.810799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:23.810799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:23.810799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:23.810799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:23.810799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:23.811115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:23.811872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:23.811872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:23.811872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:23.811872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:23.811872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:23.811872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:23.811872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:23.811872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:23.811872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:23.811872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:23.811872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:23.811872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:25.767940 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:25.768899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:25.768899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:25.768899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:25.797108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:25.797316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:25.797950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:25.797950 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:25.797950 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:25.797950 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:25.797950 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:25.797950 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:25.797950 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:25.797950 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:25.798694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:25.799367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:25.799367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:25.799367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:25.799367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:25.799367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:25.799367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:25.799367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:25.799367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:25.799367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:25.799367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:25.799367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:25.799367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:25.799367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:25.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:25.800751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:25.801691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:25.801691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:25.801691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:25.801691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:25.801691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:25.801691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:28.750401 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:28.752662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:28.752834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:28.754660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:28.755156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:28.755156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.755679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:28.755679 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.755679 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:28.755679 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:28.755679 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.755679 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:28.755679 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.755679 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:28.756720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.756720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:28.756720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:28.756720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:28.756720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:28.756720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:28.756720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:28.756720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.756720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:28.756720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:28.756720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:28.756720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.757347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:28.757347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:28.757347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:28.757347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:28.757347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:28.757347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:28.757347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:28.757347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:28.757347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.757347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:28.757347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:28.757347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.757347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:28.758103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:28.758954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.760004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:28.760004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:28.760004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.760004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:28.760004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.760004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:28.760365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:28.760365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:28.760482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:28.760544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:44:28.760827 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:28.760978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.761456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:28.761456 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.761456 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:28.761456 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:28.761456 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.761456 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:28.761456 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.761456 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:28.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:28.763243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:28.763713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:28.763713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:28.763713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:44:28.763713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:28.763713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:28.763713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.763713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:28.763713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:28.763713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:28.763713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.763713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:28.763713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:28.763713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:28.764197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:28.764197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:28.764197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:28.764197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:28.764197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:28.764197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.764197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:28.764197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:28.764197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.764197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:28.764197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.764197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:28.764578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:28.765027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:28.765545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.765545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:28.765545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:28.765545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.765545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:28.765545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.765545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:28.768465 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:28.768642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:28.768758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:28.768874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:28.769479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:28.769727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.770241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:28.770241 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.770241 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:28.770241 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:28.770241 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.770241 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:28.770241 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.770241 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:28.770961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:28.771401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:28.771401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:28.771401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.771401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:28.771401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:28.771401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.771401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:28.771401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.771401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:28.771917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.771917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:28.771917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:28.771917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:28.771917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:28.771917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:28.771917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:28.771917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:28.771917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:28.771917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.771917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:28.772339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:28.772741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.772741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:28.772741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.772741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:28.773178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:28.773734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:28.774426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.774426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:28.774426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.774426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:28.781818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:28.781983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:28.782086 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:28.782316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:28.783939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:28.786827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:28.787418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:28.787891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:28.787891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:28.787891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.787891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:28.787891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:28.787891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.787891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:28.787891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.787891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:28.787891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.787891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:28.787891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:28.788290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:28.788897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:30.768513 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:30.769553 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:30.769553 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:30.770585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:30.806578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:30.806964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:30.807299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:30.807299 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:30.807299 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:30.807299 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:30.807299 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:30.807299 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:30.807299 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:30.807299 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:30.808445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:30.808445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:30.808445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:30.808445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:30.808445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:30.808445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:30.808445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:30.808445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:30.808445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:30.808445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:30.808445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:30.808445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:30.809089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:30.809089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:30.809089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:30.809089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:30.809089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:30.809089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:30.809089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:30.809089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:30.809089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:30.809089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:30.809089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:30.809089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:30.809089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:30.809491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:30.809973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:30.810513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:30.810513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:30.810513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:30.810513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:30.810513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:30.810513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:33.752763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:33.753416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:33.753416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:33.755237 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:33.755237 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:33.755237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.755237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:33.755237 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.755237 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:33.755237 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:33.755237 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.755237 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:33.755237 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.755237 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:33.755593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.755593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:33.755593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:33.755593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:33.755593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:33.755593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:33.755593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:33.755593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.755593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:33.755593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:33.755593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:33.755593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:33.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:33.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:33.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:33.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:33.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:33.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:33.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:33.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:33.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:33.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.755836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:33.756111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.756111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:33.756349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:33.758133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.758533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:33.758533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.758533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:33.760413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:33.760664 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:33.760782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:33.760912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:44:33.761330 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:33.761514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.762043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:33.762043 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.762043 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:33.762043 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:33.762043 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.762043 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:33.762043 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.762043 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:33.762873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:33.763193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:33.763193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:33.763193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.763193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:33.763193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:33.763193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.763193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:33.763193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.763193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:33.763705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.763705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:33.763705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:33.763705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:33.763705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:33.763705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:33.763705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:33.763705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:44:33.763705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:33.763705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:33.763705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.763705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:33.763974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:33.764258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.764258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:33.764258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.764258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:33.764773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:33.765102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:33.765445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.765445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:33.765445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:33.765445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.765445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:33.765445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.765445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:33.768857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:33.769313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:33.769477 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:33.769669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:33.770175 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:33.770412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.770935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:33.770935 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.770935 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:33.770935 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:33.770935 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.770935 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:33.770935 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.770935 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:33.771781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:33.772094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:33.772094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:33.772094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.772094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:33.772094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:33.772094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.772094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:33.772094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.772094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:33.772721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.772721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:33.772721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:33.772721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:33.772721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:33.772721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:33.772721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:33.772721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:33.772721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:33.772721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.772721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:33.772962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:33.773263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.773263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:33.773263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.773263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:33.773952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:33.774343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:33.774730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.774730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:33.774730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.774730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:33.782287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:33.782630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:33.782750 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:33.782943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:33.783425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:33.783646 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:33.783787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.784352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:33.784352 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.784352 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:33.784352 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:33.784352 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.784352 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:33.784352 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.784352 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:33.785306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.785306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:33.785306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:33.785306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:33.785306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:33.785306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:33.785306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:33.785306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.785306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:33.785306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:33.785306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:33.785306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.785588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:33.785588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:33.785588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:33.785588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:33.785588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:33.785588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:33.785588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:33.785588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:33.785588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.785588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:33.785588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:33.785588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.785588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:33.785866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.785866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:33.786151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:33.786513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:33.786867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:33.786867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:33.786867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:35.769139 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:35.770056 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:35.770056 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:35.771231 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:35.817344 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:35.817344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:35.817988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:35.817988 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:35.817988 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:35.817988 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:35.817988 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:35.817988 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:35.817988 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:35.817988 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:35.819437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:35.819437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:35.819437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:35.819437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:35.819437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:35.819437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:35.819437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:35.819437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:35.819437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:35.819437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:35.819437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:35.819437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:35.819882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:35.819882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:35.819882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:35.819882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:35.819882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:35.819882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:35.819882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:35.819882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:35.819882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:35.819882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:35.819882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:35.819882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:35.819882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:35.820288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:35.820760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:35.821328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:35.821328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:35.821328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:35.821328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:35.821328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:35.821328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:38.753264 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:38.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:38.753880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:38.756441 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:38.756572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:38.756792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.757087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:38.757087 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.757087 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:38.757087 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:38.757087 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.757087 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:38.757087 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.757087 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:38.757729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.757729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:38.757729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:38.757729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:38.757729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:38.757729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:38.757729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:38.757729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.757729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:38.757729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:38.757729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:38.757729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.758042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:38.758042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:38.758042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:38.758042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:38.758042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:38.758042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:38.758042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:38.758042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:38.758042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.758042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:38.758042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:38.758042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.758042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:38.758364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.758364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:38.761226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:38.761566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.761900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:38.761900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.761900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:38.761900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:38.761900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:38.761900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:38.761900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:44:38.761900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:38.762080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.762080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:38.762080 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.762080 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:38.762080 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:38.762080 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.762080 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:38.762080 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.762080 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:38.762333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:38.762688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:38.762688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:38.762688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.762688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:38.762688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:38.762688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.762688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:38.762688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.762688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:38.762918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.762918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:38.762918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:38.762918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:38.762918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:38.762918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:38.762918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:38.762918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:44:38.762918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:38.762918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:38.762918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.762918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:38.763158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:38.763425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.763425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:38.763425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.763425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:38.763642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:38.763967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:38.764305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.764305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:38.764305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:38.764305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.764305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:38.764305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.764305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:38.769387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:38.770536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:38.770536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:38.770536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:38.770536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:38.770536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.770536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:38.770536 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.770536 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:38.770536 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:38.771046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.771801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:38.772476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:38.772476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:38.772476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:38.772476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:38.772476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:38.772476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:38.772476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:38.772476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.772476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:38.772476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:38.772476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:38.772476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.772476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:38.773035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:38.773035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:38.773035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:38.773035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:38.773035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:38.773035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:38.773035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:38.773035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.773035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:38.773035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:38.773035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.773035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:38.773035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:38.773583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.774371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:38.775154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:38.775154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.775154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:38.775154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.775154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:38.782888 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:38.783337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:38.783507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:38.783752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:38.784159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:38.784542 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:38.784759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.785609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:38.785609 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.785609 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:38.785609 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:38.785609 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.785609 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:38.785609 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.785609 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:38.787127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.787127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:38.787127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:38.787127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:38.787127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:38.787127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:38.787127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:38.787127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.787127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:38.787127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:38.787127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:38.787127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.787431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:38.787431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:38.787431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:38.787431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:38.787431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:38.787431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:38.787431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:38.787431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:38.787431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.787431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:38.787431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:38.787431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.787431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:38.787687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.787687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:38.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:38.790290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:38.790755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:38.790755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:38.790755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:40.769775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:40.770338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:40.770338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:40.771078 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:40.801545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:40.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:40.802110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:40.802110 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:40.802110 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:40.802110 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:40.802110 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:40.802110 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:40.802110 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:40.802110 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:40.802760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:40.802760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:40.802760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:40.802760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:40.802760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:40.802760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:40.802760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:40.802760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:40.802760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:40.802760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:40.802760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:40.802760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:40.803322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:40.803322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:40.803322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:40.803322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:40.803322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:40.803322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:40.803322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:40.803322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:40.803322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:40.803322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:40.803322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:40.803322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:40.803322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:40.803828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:40.804340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:40.804763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:40.804763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:40.804763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:40.804763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:40.804763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:40.804763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:43.754041 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:43.754630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:43.754630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:43.755784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:43.755784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:43.755784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.756357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:43.756357 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.756357 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:43.756357 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:43.756357 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.756357 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:43.756357 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.756357 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:43.757340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.757340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:43.757340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:43.757340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:43.757340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:43.757340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:43.757340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:43.757340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.757340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:43.757340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:43.757340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:43.757340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.757736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:43.757736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:43.757736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:43.757736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:43.757736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:43.757736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:43.757736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:43.757736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:43.757736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.757736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:43.757736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:43.757736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.757736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:43.758070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.758070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:43.758422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:43.759485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.760434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:43.760434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.760434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:43.761299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:43.761675 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:43.761798 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:43.761926 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:44:43.762263 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:43.762486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.763118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:43.763118 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.763118 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:43.763118 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:43.763118 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.763118 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:43.763118 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.763118 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:43.763876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:43.764316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:43.764316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:43.764316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.764316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:43.764316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:43.764316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.764316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:43.764316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.764316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:43.764782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.764782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:43.764782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:43.764782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:43.764782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:43.764782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:43.764782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:43.764782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:44:43.764782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:43.764782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:43.764782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.764782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:43.765642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:43.766408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:43.766794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:43.767310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:43.767310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:43.767310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:43.767310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:43.767310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:43.767310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.767310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:43.767310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:43.767310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.767310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:43.767310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.767310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:43.770311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:43.770311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:43.770311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:43.770648 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:43.770815 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:43.771056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.771581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:43.771581 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.771581 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:43.771581 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:43.771581 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.771581 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:43.771581 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.771581 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:43.772380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:43.773086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:43.773086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:43.773086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.773086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:43.773086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:43.773086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.773086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:43.773086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.773086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:43.773522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.773522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:43.773522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:43.773522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:43.773522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:43.773522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:43.773522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:43.773522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:43.773522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:43.773522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.773522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:43.773977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:43.774917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:43.775540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:43.776598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:43.776598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:43.776598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:43.776598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:43.776598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.776598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:43.776598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:43.776598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.776598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:43.776598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.776598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:43.783518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:43.783875 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:43.783998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:43.784185 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:43.784545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:43.784780 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:43.784929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.785533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:43.785533 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.785533 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:43.785533 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:43.785533 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.785533 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:43.785533 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.785533 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:43.786383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.786383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:43.786383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:43.786383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:43.786383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:43.786383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:43.786383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:43.786383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.786383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:43.786383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:43.786383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:43.786383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.786794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:43.786794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:43.786794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:43.786794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:43.786794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:43.786794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:43.786794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:43.786794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:43.786794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.786794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:43.786794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:43.786794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.786794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:43.787365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:43.787851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:43.788424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:43.788424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:43.788424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:43.788424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:43.788424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:43.788424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:45.770299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:45.771082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:45.771082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:45.772408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:45.808771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:45.809045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:45.809749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:45.809749 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:45.809749 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:45.809749 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:45.809749 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:45.809749 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:45.809749 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:45.809749 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:45.811002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:45.811002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:45.811002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:45.811002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:45.811002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:45.811002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:45.811002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:45.811002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:45.811002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:45.811002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:45.811002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:45.811002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:45.811608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:45.811608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:45.811608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:45.811608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:45.811608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:45.811608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:45.811608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:45.811608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:45.811608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:45.811608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:45.811608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:45.811608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:45.811608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:45.812264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:45.812264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:45.812775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:45.813959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:45.815058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:45.815058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:45.815058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:48.754507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:48.755110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:48.755259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:48.756387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:48.756601 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:48.756789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.757564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:48.757564 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.757564 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:48.757564 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:48.757564 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.757564 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:48.757564 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.757564 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:48.758833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.758833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:48.758833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:48.758833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:48.758833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:48.758833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:48.758833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:48.758833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.758833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:48.758833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:48.758833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:48.758833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.759137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:48.759137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:48.759137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:48.759137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:48.759137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:48.759137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:48.759137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:48.759137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:48.759137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.759137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:48.759137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:48.759137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.759137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:48.759415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.759415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:48.760071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:48.760415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.760791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:48.760791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.760791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:48.761682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:48.762061 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:48.762167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:48.762298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:44:48.762565 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:48.762732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.763152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:48.763152 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.763152 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:48.763152 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:48.763152 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.763152 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:48.763152 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.763152 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:48.763854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:48.764149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:48.764149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:48.764149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.764149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:48.764149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:48.764149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.764149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:48.764149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.764149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:48.765042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.765042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:48.765042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:48.765042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:48.765042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:48.765042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:48.765042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:48.765042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:44:48.765042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:48.765042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:48.765042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.765042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:48.765309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:48.765572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.765572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:48.765572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.765572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:48.765915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:48.766265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:48.766601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.766601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:48.766601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:48.766601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.766601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:48.766601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.766601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:48.770305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:48.770634 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:48.770769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:48.770902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:48.771176 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:48.771471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.771926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:48.771926 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.771926 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:48.771926 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:48.771926 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.771926 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:48.771926 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.771926 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:48.772661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:48.772960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:48.772960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:48.772960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.772960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:48.772960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:48.772960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.772960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:48.772960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.772960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:48.773585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.773585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:48.773585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:48.773585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:48.773585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:48.773585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:48.773585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:48.773585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:48.773585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:48.773585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.773585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:48.773828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:48.775098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:48.775445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:48.775843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.775843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:48.775843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.775843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:48.784092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:48.784435 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:48.784588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:48.784785 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:48.785299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:48.785712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:48.785982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.786945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:48.786945 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.786945 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:48.786945 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:48.786945 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.786945 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:48.786945 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.786945 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:48.788282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.788282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:48.788282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:48.788282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:48.788282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:48.788282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:48.788282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:48.788282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.788282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:48.788282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:48.788282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:48.788282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.788546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:48.788546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:48.788546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:48.788546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:48.788546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:48.788546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:48.788546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:48.788546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:48.788546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.788546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:48.788546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:48.788546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.788546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:48.788792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.788792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:48.789837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:48.790170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:48.790536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:48.790536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:48.790536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:50.771080 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:50.772302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:50.772408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:50.773358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:50.805522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:50.809236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:50.809777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:50.810239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:50.810239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:50.810239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:50.810239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:50.810239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:50.810239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:50.810239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:50.810239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:50.810239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:50.810239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:50.810239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:50.810239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:50.810239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:50.810642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:50.811236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:50.811236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:50.811236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:50.811236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:50.811236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:50.811236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:50.811236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:50.811236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:50.811236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:50.811236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:50.811236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:50.811236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:53.755173 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:53.755809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:53.756101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:53.756770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:53.757028 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:53.757306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:53.758037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.758037 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:53.758037 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:53.758037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.758037 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:53.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.758037 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:53.759666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.759666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:53.759666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:53.759666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:53.759666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:53.759666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:53.759666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:53.759666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.759666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:53.759666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:53.759666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:53.759666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.760029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:53.760029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:53.760029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:53.760029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:53.760029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:53.760029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:53.760029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:53.760029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:53.760029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.760029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:53.760029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:53.760029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.760029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:53.760277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.760277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:53.762238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:53.764402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.764762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:53.764762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.764762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:53.764762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:53.764762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:53.764762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:53.764762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:44:53.764762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:53.764936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.765418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:53.765734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.765734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:53.765734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:53.765734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:53.765734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:53.765734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:53.765734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:53.765734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:44:53.765734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:53.765734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:53.765734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.765734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:53.765967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:53.766213 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.766213 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:53.766213 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.766213 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:53.766831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:53.767151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:53.767506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.767506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:53.767506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:53.767506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.767506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:53.767506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.767506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:53.770770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:53.771066 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:53.771183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:53.771333 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:53.771586 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:53.771822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.772506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:53.772506 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.772506 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:53.772506 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:53.772506 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.772506 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:53.772506 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.772506 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:53.773366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:53.773655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:53.773655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:53.773655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.773655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:53.773655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:53.773655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.773655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:53.773655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.773655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:53.775703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.775703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:53.775703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:53.775703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:53.775703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:53.775703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:53.775703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:53.775703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:53.775703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:53.775703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.775703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:53.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:53.776485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.776485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:53.776485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.776485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:53.777010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:53.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:53.778109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.778109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:53.778109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.778109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:53.784598 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:53.785167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:53.785365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:53.785638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:53.786169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:53.786519 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:53.786861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.787518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:53.787518 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.787518 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:53.787518 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:53.787518 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.787518 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:53.787518 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.787518 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:53.788519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.788519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:53.788519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:53.788519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:53.788519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:53.788519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:53.788519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:53.788519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.788519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:53.788519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:53.788519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:53.788519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.788908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:53.788908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:53.788908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:53.788908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:53.788908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:53.788908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:53.788908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:53.788908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:53.788908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.788908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:53.788908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:53.788908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.788908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:53.789293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.789293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:53.789640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:53.790194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:53.791029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:53.791029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:53.791029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:55.773764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:55.774776 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:55.774776 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:55.775708 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:55.808894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:55.809075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:55.809549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:55.809549 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:55.809549 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:55.809549 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:55.809549 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:55.809549 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:55.809549 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:55.809549 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:55.810313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:55.810313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:55.810313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:55.810313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:55.810313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:55.810313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:55.810313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:55.810313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:55.810313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:55.810313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:55.810313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:55.810313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:55.810908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:55.810908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:55.810908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:55.810908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:55.810908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:55.810908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:55.810908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:55.810908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:55.810908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:55.810908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:55.810908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:55.810908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:55.810908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:55.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:55.812107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:55.812915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:55.812915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:55.812915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:55.812915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:55.812915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:55.812915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:58.755763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:58.756381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:58.756381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:58.756973 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:58.757137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:58.757322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.757849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:58.757849 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.757849 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:58.757849 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:58.757849 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.757849 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:58.757849 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.757849 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:58.758693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.758693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:58.758693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:58.758693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:58.758693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:58.758693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:58.758693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:58.758693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.758693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:58.758693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:58.758693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:58.758693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.759133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:58.759133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:58.759133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:58.759133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:58.759133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:58.759133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:58.759133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:58.759133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:58.759133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.759133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:58.759133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:58.759133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.759133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:58.759536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.759536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:58.759745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:58.760287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.761089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:58.761089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.761089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:58.762694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:58.762921 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:58.763033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:58.763095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:44:58.763424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:58.763631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.764102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:58.764102 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.764102 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:58.764102 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:58.764102 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.764102 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:58.764102 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.764102 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:58.764574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:58.765315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:58.765965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:58.765965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:58.765965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:44:58.765965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:58.765965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:58.765965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.765965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:58.765965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:58.765965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:58.765965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.765965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:58.765965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:58.765965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:58.766611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:58.766611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:58.766611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:58.766611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:58.766611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:58.766611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.766611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:58.766611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:58.766611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.766611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:58.766611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.766611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:58.767274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:58.767964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:58.768549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.768549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:58.768549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:58.768549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.768549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:58.768549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.768549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:58.771302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:58.771565 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:58.771672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:58.771793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:58.772049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:58.772301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.772740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:58.772740 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.772740 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:58.772740 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:58.772740 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.772740 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:58.772740 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.772740 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:44:58.773600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:44:58.774049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:58.774049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:44:58.774049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.774049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:44:58.774049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:44:58.774049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.774049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:44:58.774049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.774049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:44:58.774526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.774526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:58.774526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:58.774526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:58.774526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:58.774526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:58.774526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:58.774526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:58.774526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:58.774526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.774526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:58.774906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:58.775461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.775461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:58.775461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.775461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:58.775786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:58.776426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:58.777130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.777130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:58.777130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.777130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:44:58.785300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:44:58.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:44:58.785996 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:44:58.786316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:44:58.786851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:58.787319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:44:58.787538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.788442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:44:58.788442 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.788442 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:44:58.788442 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:44:58.788442 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.788442 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:44:58.788442 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.788442 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:44:58.789783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.789783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:44:58.789783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:58.789783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:44:58.789783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:44:58.789783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:58.789783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:58.789783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.789783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:58.789783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:58.789783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:44:58.789783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.790231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:44:58.790231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:58.790231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:44:58.790231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:58.790231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:44:58.790231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:44:58.790231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:58.790231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:44:58.790231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.790231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:44:58.790231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:44:58.790231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.790231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:44:58.790592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.790592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:44:58.791141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:44:58.791533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:44:58.791902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:44:58.791902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:44:58.791902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:00.774153 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:00.774754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:00.775271 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:00.776072 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:00.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:00.812312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:00.812932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:00.812932 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:00.812932 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:00.812932 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:00.812932 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:00.812932 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:00.812932 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:00.812932 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:00.813596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:00.813596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:00.813596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:00.813596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:00.813596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:00.813596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:00.813596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:00.813596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:00.813596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:00.813596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:00.813596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:00.813596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:00.813863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:00.813863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:00.813863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:00.813863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:00.813863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:00.813863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:00.813863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:00.813863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:00.813863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:00.813863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:00.813863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:00.813863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:00.813863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:00.814097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:00.814097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:00.814329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:00.814628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:00.814971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:00.814971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:00.814971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:03.756381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:03.757341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:03.757341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:03.757582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:03.757773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:03.757957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.758658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:03.758658 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.758658 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:03.758658 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:03.758658 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.758658 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:03.758658 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.758658 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:03.759946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.759946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:03.759946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:03.759946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:03.759946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:03.759946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:03.759946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:03.759946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.759946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:03.759946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:03.759946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:03.759946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.767202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:03.767202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:03.767202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:03.767202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:03.767202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:03.767202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:03.767202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:03.767202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:03.767202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.767202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:03.767202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:03.767202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.767202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:03.767651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:03.768177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.768812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:03.768812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:03.768812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.768812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:03.768812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.768812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:03.768812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:03.768812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:03.768812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:03.768812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:03.769174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.769634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:03.770287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.770287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:03.770287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.770287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:03.770287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:03.770287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:03.770287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:03.770287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:03.770287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:03.770287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:45:03.770287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:03.770287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:03.770653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:03.771050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.771464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:03.771976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:03.772404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:03.772404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:03.772404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:03.772586 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:03.772803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.773336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:03.773336 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.773336 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:03.773336 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:03.773336 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.773336 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:03.773336 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.773336 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:03.774224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:03.774647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:03.774647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:03.774647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.774647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:03.774647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:03.774647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.774647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:03.774647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.774647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:03.775318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.775318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:03.775318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:03.775318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:03.775318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:03.775318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:03.775318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:03.775318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:03.775318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:03.775318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.775318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:03.775649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:03.776102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.776102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:03.776102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.776102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:03.776682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:03.777425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:03.778226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.778226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:03.778226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.778226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:03.785692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:03.786023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:03.786222 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:03.786469 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:03.786950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:03.787356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:03.787597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.788442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:03.788442 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.788442 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:03.788442 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:03.788442 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.788442 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:03.788442 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.788442 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:03.789990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.789990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:03.789990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:03.789990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:03.789990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:03.789990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:03.789990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:03.789990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.789990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:03.789990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:03.789990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:03.789990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.790304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:03.790304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:03.790304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:03.790304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:03.790304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:03.790304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:03.790304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:03.790304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:03.790304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.790304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:03.790304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:03.790304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.790304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:03.790557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.790557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:03.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:03.793712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:03.794333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:03.794333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:03.794333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:05.774909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:05.775602 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:05.775695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:05.777310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:05.780130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:45:05.815262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:05.815262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:05.815520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:05.815520 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:05.815520 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:05.815520 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:05.815520 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:05.815520 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:05.815520 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:05.815520 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:05.816320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:05.816320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:05.816320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:05.816320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:05.816320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:05.816320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:05.816320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:05.816320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:05.816320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:05.816320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:05.816320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:05.816320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:05.816602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:05.816602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:05.816602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:05.816602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:05.816602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:05.816602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:05.816602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:05.816602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:05.816602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:05.816602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:05.816602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:05.816602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:05.816602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:05.816861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:05.816861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:05.817387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:05.817712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:05.818046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:05.818046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:05.818046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:08.756934 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:08.757639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:08.757639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:08.758486 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:08.758486 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:08.758626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.759106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:08.759106 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.759106 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:08.759106 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:08.759106 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.759106 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:08.759106 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.759106 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:08.760398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.760398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:08.760398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:08.760398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:08.760398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:08.760398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:08.760398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:08.760398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.760398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:08.760398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:08.760398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:08.760398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.760882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:08.760882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:08.760882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:08.760882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:08.760882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:08.760882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:08.760882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:08.760882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:08.760882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.760882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:08.760882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:08.760882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.760882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:08.761424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:08.761764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.762166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:08.762166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:08.762166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.762166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:08.762166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.762166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:08.763381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:08.763650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:08.763756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:08.763867 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:45:08.764143 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:08.764313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.764866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:08.764866 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.764866 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:08.764866 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:08.764866 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.764866 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:08.764866 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.764866 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:08.766172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:08.766523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:08.766523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:08.766523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.766523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:08.766523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:08.766523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.766523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:08.766523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.766523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:08.767538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.767538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:08.767538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:08.767538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:08.767538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:08.767538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:08.767538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:08.767538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:45:08.767538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:08.767538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:08.767538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.767538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:08.767852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:08.768115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.768115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:08.768115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.768115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:08.769667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:08.770151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:08.770719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.770719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:08.770719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:08.770719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.770719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:08.770719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.770719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:08.772025 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:08.772290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:08.772407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:08.772543 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:08.772776 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:08.772952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.773375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:08.773375 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.773375 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:08.773375 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:08.773375 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.773375 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:08.773375 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.773375 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:08.774121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:08.774555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:08.774555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:08.774555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.774555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:08.774555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:08.774555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.774555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:08.774555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.774555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:08.775046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.775046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:08.775046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:08.775046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:08.775046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:08.775046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:08.775046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:08.775046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:08.775046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:08.775046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.775046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:08.775432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:08.775818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.775818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:08.775818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.775818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:08.776178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:08.776692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:08.777281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.777281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:08.777281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.777281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:08.786098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:08.786506 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:08.786692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:08.786929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:08.787153 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:08.787351 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:08.787448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.787858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:08.787858 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.787858 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:08.787858 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:08.787858 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.787858 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:08.787858 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.787858 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:08.789229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.789229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:08.789229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:08.789229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:08.789229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:08.789229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:08.789229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:08.789229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.789229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:08.789229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:08.789229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:08.789229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.789775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:08.789775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:08.789775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:08.789775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:08.789775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:08.789775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:08.789775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:08.789775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:08.789775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.789775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:08.789775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:08.789775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.789775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:08.790341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.790341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:08.791105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:08.791798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:08.792697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:08.792697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:08.792697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:10.775590 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:10.776586 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:10.776653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:10.777896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:10.817175 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:10.817683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:10.818606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:10.818606 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:10.818606 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:10.818606 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:10.818606 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:10.818606 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:10.818606 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:10.818606 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:10.819667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:10.819667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:10.819667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:10.819667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:10.819667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:10.819667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:10.819667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:10.819667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:10.819667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:10.819667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:10.819667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:10.819667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:10.820081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:10.820081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:10.820081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:10.820081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:10.820081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:10.820081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:10.820081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:10.820081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:10.820081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:10.820081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:10.820081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:10.820081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:10.820081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:10.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:10.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:10.820713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:10.821189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:10.821939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:10.821939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:10.821939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:13.757504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:13.758074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:13.758277 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:13.759340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:13.759563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:13.759763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.760471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:13.760471 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.760471 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:13.760471 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:13.760471 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.760471 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:13.760471 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.760471 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:13.761518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.761518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:13.761518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:13.761518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:13.761518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:13.761518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:13.761518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:13.761518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.761518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:13.761518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:13.761518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:13.761518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.761816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:13.761816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:13.761816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:13.761816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:13.761816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:13.761816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:13.761816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:13.761816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:13.761816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.761816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:13.761816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:13.761816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.761816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:13.762093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.762093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:13.762600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:13.763143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.763784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:13.763784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.763784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:13.763784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:13.763927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:13.764439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:13.764439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:45:13.764566 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:13.764730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.765233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:13.765233 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.765233 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:13.765233 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:13.765233 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.765233 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:13.765233 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.765233 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:13.766087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:13.766630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:13.766630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:13.766630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.766630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:13.766630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:13.766630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.766630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:13.766630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.766630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:13.766933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.766933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:13.766933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:13.766933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:13.766933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:13.766933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:13.766933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:13.766933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:45:13.766933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:13.766933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:13.766933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.766933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:13.767338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:13.767734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.767734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:13.767734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.767734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:13.767866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:13.768346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:13.768859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.768859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:13.768859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:13.768859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.768859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:13.768859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.768859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:13.772368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:13.772594 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:13.772703 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:13.772823 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:13.773351 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:13.773767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.774962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:13.774962 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.774962 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:13.774962 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:13.774962 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.774962 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:13.774962 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.774962 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:13.775404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:13.775836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:13.775836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:13.775836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.775836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:13.775836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:13.775836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.775836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:13.775836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.775836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:13.776329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.776329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:13.776329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:13.776329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:13.776329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:13.776329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:13.776329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:13.776329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:13.776329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:13.776329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.776329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:13.776842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:13.777238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.777238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:13.777238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.777238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:13.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:13.777870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:13.778394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.778394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:13.778394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.778394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:13.786456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:13.786769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:13.786955 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:13.787243 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:13.788313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:13.788313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:13.788313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.790378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:13.790378 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.790378 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:13.790378 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:13.790378 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.790378 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:13.790378 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.790378 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:13.790714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.790714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:13.790714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:13.790714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:13.790714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:13.790714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:13.790714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:13.790714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.790714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:13.790714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:13.790714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:13.790714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.790984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:13.790984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:13.790984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:13.790984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:13.790984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:13.790984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:13.790984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:13.790984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:13.790984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.790984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:13.790984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:13.790984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.790984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:13.791286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.791286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:13.792342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:13.792698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:13.793064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:13.793064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:13.793064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:15.776636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:15.777141 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:15.779732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:15.779732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:15.818850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:15.819052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:15.819579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:15.819579 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:15.819579 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:15.819579 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:15.819579 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:15.819579 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:15.819579 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:15.819579 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:15.820390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:15.820390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:15.820390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:15.820390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:15.820390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:15.820390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:15.820390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:15.820390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:15.820390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:15.820390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:15.820390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:15.820390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:15.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:15.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:15.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:15.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:15.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:15.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:15.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:15.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:15.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:15.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:15.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:15.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:15.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:15.820947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:15.820947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:15.821289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:15.821821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:15.822179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:15.822179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:15.822179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:18.758267 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:18.758833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:18.758963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:18.759776 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:18.759934 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:18.760044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.760730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:18.760730 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.760730 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:18.760730 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:18.760730 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.760730 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:18.760730 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.760730 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:18.762482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.762482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:18.762482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:18.762482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:18.762482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:18.762482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:18.762482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:18.762482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.762482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:18.762482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:18.762482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:18.762482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.762899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:18.762899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:18.762899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:18.762899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:18.762899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:18.762899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:18.762899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:18.762899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:18.762899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.762899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:18.762899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:18.762899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.762899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:18.763284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.763284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:18.764957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:18.765444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.766627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:18.766627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.766627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:18.766627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:18.766627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:18.766627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:18.766627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:45:18.768231 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:18.768392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.769012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:18.769012 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.769012 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:18.769012 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:18.769012 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.769012 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:18.769012 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.769012 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:18.770157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:18.770860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:18.770860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:18.770860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.770860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:18.770860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:18.770860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.770860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:18.770860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.770860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:18.771422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.771422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:18.771422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:18.771422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:18.771422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:18.771422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:18.771422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:18.771422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:45:18.771422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:18.771422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:18.771422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.771422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:18.771675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:18.771945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.771945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:18.771945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.771945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:18.772961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:18.773287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:18.773611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.773611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:18.773611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:18.773611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.773611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:18.773611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.773611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:18.773611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:18.773611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:18.773811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:18.773841 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:18.774467 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:18.774773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.775297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:18.775297 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.775297 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:18.775297 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:18.775297 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.775297 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:18.775297 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.775297 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:18.776200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:18.776528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:18.776528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:18.776528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.776528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:18.776528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:18.776528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.776528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:18.776528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.776528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:18.776703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.776703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:18.776703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:18.776703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:18.776703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:18.776703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:18.776703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:18.776703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:18.776703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:18.776703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.776703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:18.776933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:18.777191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.777191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:18.777191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.777191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:18.777577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:18.777926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:18.778283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.778283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:18.778283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.778283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:18.787617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:18.788450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:18.788589 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:18.789008 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:18.789581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:18.789854 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:18.790008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.790737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:18.790737 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.790737 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:18.790737 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:18.790737 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.790737 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:18.790737 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.790737 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:18.792226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.792226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:18.792226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:18.792226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:18.792226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:18.792226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:18.792226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:18.792226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.792226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:18.792226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:18.792226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:18.792226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.792624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:18.792624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:18.792624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:18.792624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:18.792624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:18.792624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:18.792624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:18.792624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:18.792624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.792624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:18.792624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:18.792624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.792624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:18.793009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.793009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:18.793484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:18.793919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:18.794440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:18.794440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:18.794440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:20.776929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:20.777729 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:20.777729 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:20.778428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:20.813684 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:20.813955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:20.814749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:20.814749 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:20.814749 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:20.814749 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:20.814749 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:20.814749 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:20.814749 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:20.814749 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:20.815753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:20.815753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:20.815753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:20.815753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:20.815753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:20.815753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:20.815753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:20.815753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:20.815753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:20.815753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:20.815753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:20.815753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:20.816124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:20.816124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:20.816124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:20.816124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:20.816124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:20.816124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:20.816124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:20.816124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:20.816124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:20.816124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:20.816124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:20.816124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:20.816124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:20.816563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:20.816563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:20.816911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:20.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:20.817845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:20.817845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:20.817845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:23.759000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:23.761211 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:23.761211 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:23.761616 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:23.762001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:23.762284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.763178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:23.763178 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.763178 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:23.763178 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:23.763178 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.763178 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:23.763178 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.763178 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:23.766512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.766512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:23.766512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:23.766512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:23.766512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:23.766512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:23.766512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:23.766512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.766512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:23.766512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:23.766512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:23.766512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.767096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:23.767096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:23.767096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:23.767096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:23.767096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:23.767096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:23.767096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:23.767096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:23.767096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.767096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:23.767096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:23.767096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.767096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:23.767671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:23.768331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.769122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:23.769122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:23.769122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.769122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:23.769122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.769122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:23.769122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:23.769122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:23.769122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:23.769122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:45:23.769612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:23.769767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.770603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:23.770603 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.770603 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:23.770603 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:23.770603 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.770603 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:23.770603 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.770603 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:23.771982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:23.772472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:23.772472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:23.772472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.772472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:23.772472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:23.772472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.772472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:23.772472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.772472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:23.773399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.773399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:23.773399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:23.773399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:23.773399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:23.773399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:23.773399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:23.773399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:45:23.773399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:23.773399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:23.773399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.773399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:23.773763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:23.774262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.774262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:23.774262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.774262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:23.775359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:23.775850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:23.776382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.776382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:23.776382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:23.776382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.776382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:23.776382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.776382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:23.776382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:23.776382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:23.776382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:23.776697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:23.776697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:23.777041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.777738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:23.777738 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.777738 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:23.777738 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:23.777738 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.777738 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:23.777738 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.777738 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:23.778907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:23.779332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:23.779332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:23.779332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.779332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:23.779332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:23.779332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.779332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:23.779332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.779332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:23.779590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.779590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:23.779590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:23.779590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:23.779590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:23.779590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:23.779590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:23.779590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:23.779590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:23.779590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.779590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:23.779901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:23.780279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:23.780651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:23.781207 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:23.781207 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:23.781207 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:23.781207 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:23.781207 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.781207 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:23.781207 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:23.781207 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.781207 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:23.781207 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.781207 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:23.788114 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:23.789544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:23.789544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:23.789544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:23.790396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:23.790396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:23.790396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.791282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:23.791282 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.791282 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:23.791282 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:23.791282 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.791282 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:23.791282 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.791282 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:23.792637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.792637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:23.792637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:23.792637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:23.792637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:23.792637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:23.792637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:23.792637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.792637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:23.792637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:23.792637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:23.792637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.793338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:23.793338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:23.793338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:23.793338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:23.793338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:23.793338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:23.793338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:23.793338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:23.793338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.793338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:23.793338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:23.793338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.793338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:23.793945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.793945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:23.794599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:23.795861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:23.796768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:23.796768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:23.796768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:25.777581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:25.779287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:25.779287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:25.780240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:25.815248 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:25.815466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:25.816106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:25.816106 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:25.816106 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:25.816106 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:25.816106 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:25.816106 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:25.816106 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:25.816106 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:25.817000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:25.817000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:25.817000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:25.817000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:25.817000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:25.817000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:25.817000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:25.817000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:25.817000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:25.817000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:25.817000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:25.817000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:25.817686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:25.817686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:25.817686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:25.817686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:25.817686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:25.817686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:25.817686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:25.817686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:25.817686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:25.817686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:25.817686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:25.817686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:25.817686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:25.818418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:25.819289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:25.820466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:25.820466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:25.820466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:25.820466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:25.820466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:25.820466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:28.760738 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:28.761409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:28.761551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:28.762556 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:28.763124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:28.763124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.764167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:28.764167 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.764167 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:28.764167 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:28.764167 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.764167 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:28.764167 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.764167 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:28.765692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.765692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:28.765692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:28.765692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:28.765692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:28.765692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:28.765692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:28.765692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.765692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:28.765692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:28.765692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:28.765692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.765970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:28.765970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:28.765970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:28.765970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:28.765970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:28.765970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:28.765970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:28.765970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:28.765970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.765970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:28.765970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:28.765970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.765970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:28.766326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.766326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:28.767451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:28.767904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.768423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:28.768423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.768423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:28.768971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:28.769082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:28.769201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:28.769356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:45:28.769709 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:28.771523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.772353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:28.772948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.772948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:28.772948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:28.772948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:28.772948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:28.772948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:28.772948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:28.772948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:45:28.772948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:28.772948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:28.772948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.772948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:28.773467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:28.773970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:28.774491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:28.775406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:28.775406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:28.775406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:28.775406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:28.775406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:28.775406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.775406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:28.775406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:28.775406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.775406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:28.775406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.775406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:28.775972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:28.776259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:28.776400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:28.776519 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:28.776867 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:28.777183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.777775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:28.777775 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.777775 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:28.777775 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:28.777775 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.777775 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:28.777775 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.777775 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:28.778485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:28.779003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:28.779003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:28.779003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.779003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:28.779003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:28.779003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.779003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:28.779003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.779003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:28.779413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.779413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:28.779413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:28.779413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:28.779413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:28.779413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:28.779413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:28.779413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:28.779413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:28.779413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.779413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:28.779844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:28.780368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:28.780854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:28.781531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:28.781531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:28.781531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:28.781531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:28.781531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.781531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:28.781531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:28.781531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.781531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:28.781531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.781531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:28.788485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:28.788748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:28.788850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:28.788998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:28.789406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:28.789695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:28.789872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.790539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:28.790539 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.790539 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:28.790539 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:28.790539 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.790539 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:28.790539 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.790539 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:28.791881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.791881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:28.791881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:28.791881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:28.791881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:28.791881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:28.791881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:28.791881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.791881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:28.791881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:28.791881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:28.791881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.792375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:28.792375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:28.792375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:28.792375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:28.792375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:28.792375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:28.792375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:28.792375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:28.792375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.792375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:28.792375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:28.792375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.792375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:28.792876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.792876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:28.793551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:28.794277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:28.794966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:28.794966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:28.794966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:30.779285 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:30.780157 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:30.780157 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:30.780945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:30.809929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:30.810411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:30.811259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:30.811259 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:30.811259 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:30.811259 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:30.811259 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:30.811259 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:30.811259 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:30.811259 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:30.812094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:30.812094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:30.812094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:30.812094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:30.812094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:30.812094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:30.812094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:30.812094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:30.812094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:30.812094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:30.812094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:30.812094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:30.812598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:30.812598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:30.812598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:30.812598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:30.812598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:30.812598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:30.812598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:30.812598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:30.812598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:30.812598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:30.812598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:30.812598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:30.812598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:30.812958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:30.813426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:30.813955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:30.813955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:30.813955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:30.813955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:30.813955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:30.813955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:33.762367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:33.762367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:33.762367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:33.762977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:33.763326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:33.763510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.764276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:33.764276 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.764276 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:33.764276 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:33.764276 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.764276 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:33.764276 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.764276 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:33.765764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.765764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:33.765764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:33.765764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:33.765764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:33.765764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:33.765764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:33.765764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.765764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:33.765764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:33.765764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:33.765764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.766573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:33.766573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:33.766573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:33.766573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:33.766573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:33.766573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:33.766573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:33.766573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:33.766573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.766573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:33.766573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:33.766573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.766573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:33.766951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.766951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:33.767576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:33.768179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.768700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:33.768700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.768700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:33.769179 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:33.769500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:33.769623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:33.769740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:45:33.770133 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:33.770405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.770891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:33.770891 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.770891 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:33.770891 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:33.770891 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.770891 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:33.770891 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.770891 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:33.771901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:33.772596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:33.772596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:33.772596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.772596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:33.772596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:33.772596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.772596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:33.772596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.772596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:33.773188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.773188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:33.773188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:33.773188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:33.773188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:33.773188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:33.773188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:33.773188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:45:33.773188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:33.773188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:33.773188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.773188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:33.773584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:33.773999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.773999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:33.773999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.773999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:33.774442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:33.774902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:33.775413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.775413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:33.775413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:33.775413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.775413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:33.775413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.775413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:33.776379 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:33.776650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:33.776767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:33.776907 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:33.777313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:33.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:33.778143 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.778143 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:33.778143 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:33.778143 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.778143 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:33.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:33.778879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:33.779327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:33.779327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:33.779327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.779327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:33.779327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:33.779327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.779327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:33.779327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.779327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:33.779718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.779718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:33.779718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:33.779718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:33.779718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:33.779718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:33.779718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:33.779718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:33.779718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:33.779718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.779718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:33.780037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:33.780456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.780456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:33.780456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.780456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:33.780759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:33.781252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:33.781773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.781773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:33.781773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.781773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:33.788928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:33.789220 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:33.789368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:33.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:33.789899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:33.790286 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:33.790514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:33.791250 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.791250 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:33.791250 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:33.791250 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.791250 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:33.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:33.792047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.792047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:33.792047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:33.792047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:33.792047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:33.792047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:33.792047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:33.792047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.792047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:33.792047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:33.792047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:33.792047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:33.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:33.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:33.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:33.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:33.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:33.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:33.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:33.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:33.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:33.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:33.793255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:33.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:33.794843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:33.794843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:33.794843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:33.794843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:33.794843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:33.794843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:35.779596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:35.780453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:35.780453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:35.781570 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:35.815036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:35.815231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:35.815821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:35.815821 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:35.815821 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:35.815821 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:35.815821 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:35.815821 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:35.815821 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:35.815821 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:35.816655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:35.816655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:35.816655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:35.816655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:35.816655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:35.816655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:35.816655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:35.816655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:35.816655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:35.816655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:35.816655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:35.816655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:35.817307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:35.817307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:35.817307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:35.817307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:35.817307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:35.817307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:35.817307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:35.817307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:35.817307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:35.817307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:35.817307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:35.817307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:35.817307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:35.817912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:35.818705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:35.819689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:35.819689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:35.819689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:35.819689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:35.819689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:35.819689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:38.762141 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:38.763240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:38.763240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:38.763628 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:38.764009 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:38.764345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.765241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:38.765241 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.765241 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:38.765241 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:38.765241 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.765241 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:38.765241 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.765241 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:38.766165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.766165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:38.766165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:38.766165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:38.766165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:38.766165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:38.766165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:38.766165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.766165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:38.766165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:38.766165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:38.766165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.766804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:38.766804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:38.766804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:38.766804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:38.766804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:38.766804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:38.766804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:38.766804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:38.766804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.766804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:38.766804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:38.766804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.766804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:38.767428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:38.768166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.769723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:38.769723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:38.769723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.769723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:38.769723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.769723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:38.769723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:38.769940 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:38.769940 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:38.770058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:45:38.770380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:38.770775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.771309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:38.771309 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.771309 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:38.771309 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:38.771309 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.771309 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:38.771309 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.771309 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:38.772036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:38.772572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:38.772572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:38.772572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.772572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:38.772572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:38.772572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.772572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:38.772572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.772572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:38.772866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.772866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:38.772866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:38.772866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:38.772866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:38.772866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:38.772866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:38.772866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:45:38.772866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:38.772866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:38.772866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.772866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:38.773230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:38.773607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.773607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:38.773607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.773607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:38.773969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:38.774438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:38.774904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.774904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:38.774904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:38.774904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.774904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:38.774904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.774904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:38.776720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:38.777827 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:38.777929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:38.778688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:38.779105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:38.779513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.781220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:38.781220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.781220 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:38.781220 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:38.781220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.781220 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:38.781220 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.781220 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:38.781942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:38.782355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:38.782355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:38.782355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.782355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:38.782355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:38.782355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.782355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:38.782355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.782355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:38.782796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.782796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:38.782796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:38.782796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:38.782796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:38.782796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:38.782796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:38.782796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:38.782796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:38.782796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.782796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:38.783120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:38.783507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.783507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:38.783507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.783507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:38.783816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:38.784318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:38.784765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.784765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:38.784765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.784765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:38.789370 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:38.789582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:38.789686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:38.789831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:38.790092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:38.790314 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:38.790453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.790903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:38.790903 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.790903 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:38.790903 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:38.790903 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.790903 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:38.790903 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.790903 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:38.791688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.791688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:38.791688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:38.791688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:38.791688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:38.791688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:38.791688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:38.791688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.791688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:38.791688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:38.791688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:38.791688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.792018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:38.792018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:38.792018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:38.792018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:38.792018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:38.792018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:38.792018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:38.792018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:38.792018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.792018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:38.792018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:38.792018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.792018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:38.792408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.792408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:38.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:38.793043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:38.793517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:38.793517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:38.793517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:40.780346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:40.781287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:40.781287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:40.781970 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:40.810041 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:40.810348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:40.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:40.811000 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:40.811000 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:40.811000 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:40.811000 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:40.811000 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:40.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:40.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:40.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:40.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:40.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:40.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:40.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:40.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:40.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:40.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:40.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:40.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:40.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:40.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:40.812532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:40.812532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:40.812532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:40.812532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:40.812532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:40.812532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:40.812532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:40.812532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:40.812532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:40.812532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:40.812532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:40.812532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:40.812532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:40.813293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:40.813843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:40.814389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:40.814389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:40.814389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:40.814389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:40.814389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:40.814389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:43.762740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:43.763569 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:43.763569 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:43.764428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:43.764613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:43.764736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.765372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:43.765372 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.765372 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:43.765372 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:43.765372 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.765372 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:43.765372 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.765372 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:43.766906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.766906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:43.766906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:43.766906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:43.766906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:43.766906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:43.766906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:43.766906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.766906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:43.766906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:43.766906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:43.766906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.767514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:43.767514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:43.767514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:43.767514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:43.767514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:43.767514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:43.767514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:43.767514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:43.767514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.767514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:43.767514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:43.767514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.767514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:43.768072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:43.768783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.769724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:43.769724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:43.769724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.769724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:43.769724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.769724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:43.770014 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:43.770260 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:43.770430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:43.770599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:45:43.770891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:43.771036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.771490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:43.771490 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.771490 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:43.771490 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:43.771490 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.771490 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:43.771490 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.771490 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:43.772467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:43.772883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:43.772883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:43.772883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.772883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:43.772883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:43.772883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.772883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:43.772883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.772883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:43.773394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.773394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:43.773394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:43.773394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:43.773394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:43.773394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:43.773394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:43.773394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:45:43.773394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:43.773394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:43.773394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.773394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:43.773775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:43.774140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.774140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:43.774140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.774140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:43.774536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:43.774983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:43.775482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.775482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:43.775482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:43.775482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.775482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:43.775482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.775482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:43.777868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:43.778145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:43.778315 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:43.778459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:43.778767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:43.778973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.779466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:43.779466 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.779466 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:43.779466 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:43.779466 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.779466 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:43.779466 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.779466 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:43.780362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:43.780807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:43.780807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:43.780807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.780807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:43.780807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:43.780807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.780807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:43.780807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.780807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:43.781234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.781234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:43.781234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:43.781234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:43.781234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:43.781234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:43.781234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:43.781234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:43.781234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:43.781234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.781234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:43.781607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:43.782030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.782030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:43.782030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.782030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:43.782570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:43.783260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:43.783794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.783794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:43.783794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.783794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:43.789689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:43.790034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:43.790156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:43.790356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:43.790639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:43.790854 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:43.790992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.791495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:43.791495 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.791495 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:43.791495 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:43.791495 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.791495 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:43.791495 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.791495 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:43.792441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.792441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:43.792441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:43.792441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:43.792441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:43.792441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:43.792441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:43.792441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.792441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:43.792441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:43.792441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:43.792441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.792822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:43.792822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:43.792822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:43.792822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:43.792822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:43.792822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:43.792822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:43.792822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:43.792822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.792822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:43.792822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:43.792822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.792822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:43.793166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.793166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:43.793456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:43.793892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:43.794396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:43.794396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:43.794396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:45.771297 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:45.772075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:45.772075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:45.780939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:45.782632 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 07 15:45:45.782856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105503) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 07 15:45:45.783030 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:45:45.813106 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, resource={'id': '8b456c85-5e2d-47f8-9886-ebf65932cf32', 'name': 'private', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4a00c537-ed4f-405c-b241-82e7bc2cf3fa', '9fc89727-343d-403a-acd7-db6b4950e7b1'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:25:31Z', 'updated_at': '2026-05-07T15:25:34Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 42799}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.813448 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.813990 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-b0f8152d-626b-440e-9383-919ec905ab38, external_ids_update=None, columns={'tag': 42799}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.814136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.814856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:45.814856 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:45.814856 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:45.814856 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:45.814856 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:45.814856 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:45.814856 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.814856 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:45.815667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.815667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:45.815667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:45:45.815667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:45:45.815667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:45:45.815667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:45:45.815667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:45:45.815667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:45.815667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:45.815667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:45.815667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:45.816415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:45.817195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:45.817195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.817195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:45.817195 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:45:45.838403 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, resource={'id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'name': 'public', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6aa26064-0a62-428b-a8ed-fef52fc73984', 'adf79775-4dfa-4110-bca0-59be9560299c'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-07T15:25:37Z', 'updated_at': '2026-05-07T15:25:45Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.838403 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.839026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-2b8b3cbc-130c-4894-94e1-ab0729ec7077, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.839398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.840028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:45.840028 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:45.840028 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:45.840028 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:45.840028 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:45.840028 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:45.840028 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.840028 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:45.840829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.840829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:45.840829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:45:45.840829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:45:45.840829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:45:45.840829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:45:45.840829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:45:45.840829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:45.840829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:45.840829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:45.840829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:45.841604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:45.842328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:45.842328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.842328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:45.842328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:45:45.860775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, resource={'id': '71aea053-d575-4d13-afc5-e6241ad6a15f', 'name': 'admin_net', 'tenant_id': '74d9c701d5cb4de6a15233057b01e413', 'project_id': '74d9c701d5cb4de6a15233057b01e413', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8e8a2cf7-97a7-4d62-ae4a-fa6c2139e811'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:30:51Z', 'updated_at': '2026-05-07T15:30:52Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 51150}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.861119 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.861662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d5042c49-4c90-465a-b356-c18cf966a58b, external_ids_update=None, columns={'tag': 51150}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.861868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.862632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:45.862632 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:45.862632 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:45.862632 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:45.862632 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:45.862632 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:45.862632 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.862632 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:45.863402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.863402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:45.863402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:45:45.863402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:45:45.863402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:45:45.863402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:45:45.863402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:45:45.863402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:45.863402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:45.863402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:45.863402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:45.864001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:45.864747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:45.864747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.864747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:45.864747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:45:45.884302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, resource={'id': '4fb571ad-ed2f-4edf-a199-cb52ff142698', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'tenant_id': '91e1e016e0f8434890282eb48ed58760', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2e6720c7-3159-41b9-80a2-fa6cf6a18808'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:31:26Z', 'updated_at': '2026-05-07T15:31:27Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 29129}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.884588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.884887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13b8a284-1038-44ba-86e4-667f771dfae2, external_ids_update=None, columns={'tag': 29129}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.884994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.885495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:45.885495 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:45.885495 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:45.885495 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:45.885495 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:45.885495 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:45.885495 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.885495 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:45.886258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.886258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:45.886258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:45:45.886258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:45:45.886258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:45:45.886258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:45:45.886258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:45:45.886258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:45.886258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:45.886258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:45.886258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:45.886795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:45.887419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:45.887419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.887419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:45.887419 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:45:45.952914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, resource={'id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'enable_snat': True}, 'gw_port_id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-jseci', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'ip_address': '172.24.5.213', 'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984'}, {'ip_address': '2001:db8::15a', 'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 3, 'tenant_id': '91e1e016e0f8434890282eb48ed58760'}, resource_type=routers, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.953164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.953164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.953353 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:45.953515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.954770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:45.954770 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:45.954770 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:45.954770 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:45.954770 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:45.954770 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:45.954770 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.954770 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:45.954770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Unable to update router 0938bfb8-1052-49e2-bd69-e5fda45e8a2f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:45.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:45:45.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:45:45.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:45:45.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:45:45.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 07 15:45:45.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 07 15:45:45.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 07 15:45:45.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 07 15:45:45.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:45.956714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:45.957420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:45.957420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:45.957420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:45.957420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:45.957420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:45.957420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:45.957420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:45.957420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:45.957420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:45.957420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:45:46.066959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, resource={'id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'name': '', 'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'project_id': '', 'mac_address': 'fa:16:3e:f6:db:65', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000172501', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:46.067310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '6aa26064-0a62-428b-a8ed-fef52fc73984 adf79775-4dfa-4110-bca0-59be9560299c', 'neutron:network_name': 'neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.213/24', '2001:db8::15a/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:46.067633 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=b30ffafe-8df3-4064-bcbf-e3d1daad619c, lrouter_port=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:46.067890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:46.068647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:46.068647 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:46.068647 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:46.068647 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:46.068647 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:46.068647 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:46.068647 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:46.068647 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:46.069803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:46.069803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:46.069803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:45:46.069803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:45:46.069803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:45:46.069803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:45:46.069803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 07 15:45:46.069803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:46.069803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:46.069803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:46.069803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:46.070349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:46.070705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:46.070705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:46.070705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:46.070705 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_for_inconsistencies finished in 0.298 seconds
May 07 15:45:46.070705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:46.070705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:46.071434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:46.267830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:46.268346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:46.269271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:46.269271 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:46.269271 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:46.269271 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:46.269271 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:46.269271 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:46.269271 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:46.269271 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:46.270261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.199 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:46.270261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:46.270261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:46.270261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:46.270261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:46.270261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:46.270261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:46.270261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:46.270261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:46.270261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:46.270261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:46.270261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:46.270890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:46.270890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:46.270890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:46.270890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:46.270890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:46.270890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:46.270890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:46.270890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:46.270890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:46.270890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:46.270890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:46.270890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:46.270890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:46.271581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:46.272571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:46.273638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:46.273638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:46.273638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:46.273638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:46.273638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:46.273638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:48.763413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:48.764298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:48.764298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:48.765228 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:48.765565 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:48.765807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.766712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:48.766712 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.766712 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:48.766712 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:48.766712 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.766712 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:48.766712 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.766712 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:48.768197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.768197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:48.768197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:48.768197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:48.768197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:48.768197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:48.768197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:48.768197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.768197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:48.768197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:48.768197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:48.768197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.768547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:48.768547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:48.768547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:48.768547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:48.768547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:48.768547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:48.768547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:48.768547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:48.768547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.768547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:48.768547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:48.768547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.768547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:48.768901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.768901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:48.769810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:48.770272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.770757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:48.770757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.770757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:48.770757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:48.770757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:48.770757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:48.770945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:45:48.771123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:48.771325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.771867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:48.771867 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.771867 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:48.771867 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:48.771867 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.771867 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:48.771867 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.771867 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:48.772919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:48.773334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:48.773334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:48.773334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.773334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:48.773334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:48.773334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.773334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:48.773334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.773334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:48.773765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.773765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:48.773765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:48.773765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:48.773765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:48.773765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:48.773765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:48.773765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:45:48.773765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:48.773765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:48.773765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.773765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:48.774104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:48.774469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.774469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:48.774469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.774469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:48.774798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:48.775356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:48.775827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.775827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:48.775827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:48.775827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.775827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:48.775827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.775827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:48.778286 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:48.778580 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:48.778693 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:48.778829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:48.779228 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:48.779591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.780315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:48.780315 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.780315 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:48.780315 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:48.780315 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.780315 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:48.780315 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.780315 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:48.781070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:48.781697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:48.781697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:48.781697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.781697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:48.781697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:48.781697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.781697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:48.781697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.781697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:48.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:48.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:48.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:48.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:48.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:48.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:48.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:48.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:48.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:48.782450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:48.782861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.782861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:48.782861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.782861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:48.782980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:48.783605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:48.784332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.784332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:48.784332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.784332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:48.790033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:48.790412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:48.790578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:48.790817 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:48.792161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:48.792161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:48.792161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.792161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:48.792161 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.792161 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:48.792161 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:48.792161 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.792161 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:48.792161 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.792161 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:48.793445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.793445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:48.793445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:48.793445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:48.793445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:48.793445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:48.793445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:48.793445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.793445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:48.793445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:48.793445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:48.793445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.793751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:48.793751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:48.793751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:48.793751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:48.793751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:48.793751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:48.793751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:48.793751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:48.793751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.793751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:48.793751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:48.793751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.793751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:48.794057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.794057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:48.794935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:48.795435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:48.795931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:48.795931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:48.795931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:51.070710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:51.072501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:51.072501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:51.073140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:51.101915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:51.102446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:51.103331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:51.103331 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:51.103331 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:51.103331 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:51.103331 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:51.103331 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:51.103331 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:51.103331 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:51.104519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:51.104519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:51.104519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:51.104519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:51.104519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:51.104519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:51.104519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:51.104519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:51.104519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:51.104519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:51.104519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:51.104519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:51.104945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:51.104945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:51.104945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:51.104945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:51.104945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:51.104945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:51.104945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:51.104945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:51.104945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:51.104945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:51.104945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:51.104945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:51.104945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:51.105389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:51.105389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:51.107556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:51.108131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:51.108844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:51.108844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:51.108844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:53.764018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:53.765073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:53.765073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:53.765560 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:53.765560 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:53.765746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.766274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:53.766274 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.766274 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:53.766274 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:53.766274 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.766274 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:53.766274 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.766274 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:53.767017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.767017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:53.767017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:53.767017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:53.767017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:53.767017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:53.767017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:53.767017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.767017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:53.767017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:53.767017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:53.767017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.767530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:53.767530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:53.767530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:53.767530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:53.767530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:53.767530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:53.767530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:53.767530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:53.767530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.767530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:53.767530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:53.767530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.767530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:53.768028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:53.768762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.769336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:53.769336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:53.769336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.769336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:53.769336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.769336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:53.770495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:53.770754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:53.770810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:53.771041 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:45:53.771662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:53.771842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.772291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:53.772291 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.772291 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:53.772291 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:53.772291 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.772291 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:53.772291 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.772291 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:53.772833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:53.773510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:53.774055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:53.774055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:53.774055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:45:53.774055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:53.774055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:53.774055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.774055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:53.774055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:53.774055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:53.774055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.774055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:53.774055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:53.774055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:53.774477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:53.774477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:53.774477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:53.774477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:53.774477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:53.774477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.774477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:53.774477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:53.774477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.774477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:53.774477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.774477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:53.774814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:53.775291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:53.775777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.775777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:53.775777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:53.775777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.775777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:53.775777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.775777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:53.778648 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:53.778983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:53.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:53.779390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:53.780288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:53.780288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.780791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:53.780791 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.780791 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:53.780791 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:53.780791 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.780791 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:53.780791 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.780791 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:53.782168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:53.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:53.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:53.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:53.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:53.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:53.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:53.783845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.783845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:53.783845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:53.783845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:53.783845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:53.783845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:53.783845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:53.783845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:53.783845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:53.783845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.783845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:53.784241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:53.784620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.784620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:53.784620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.784620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:53.785875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:53.786421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:53.786926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.786926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:53.786926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.786926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:53.790345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:53.790590 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:53.790697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:53.790842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:53.791110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:53.791330 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:53.791469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.791918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:53.791918 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.791918 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:53.791918 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:53.791918 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.791918 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:53.791918 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.791918 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:53.792775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.792775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:53.792775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:53.792775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:53.792775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:53.792775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:53.792775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:53.792775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.792775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:53.792775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:53.792775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:53.792775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.793272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:53.793272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:53.793272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:53.793272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:53.793272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:53.793272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:53.793272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:53.793272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:53.793272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.793272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:53.793272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:53.793272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.793272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:53.793619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.793619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:53.793745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:53.794328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:53.794821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:53.794821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:53.794821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:56.071348 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:56.071969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:56.071969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:56.073200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:56.109444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:56.109800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:56.110425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:56.110425 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:56.110425 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:56.110425 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:56.110425 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:56.110425 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:56.110425 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:56.110425 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:56.111169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:56.111169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:56.111169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:56.111169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:56.111169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:56.111169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:56.111169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:56.111169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:56.111169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:56.111169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:56.111169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:56.111169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:56.112368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:56.112368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:56.112368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:56.112368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:56.112368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:56.112368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:56.112368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:56.112368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:56.112368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:56.112368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:56.112368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:56.112368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:56.112368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:56.113182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:56.114136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:56.116158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:56.116158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:56.116158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:56.116158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:56.116158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:56.116158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:58.764912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:58.765744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:58.765744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:58.766403 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:58.766534 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:58.766705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:58.767198 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.767198 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:58.767198 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:58.767198 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.767198 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:58.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.767198 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:58.768957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.768957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:58.768957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:58.768957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:58.768957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:58.768957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:58.768957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:58.768957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.768957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:58.768957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:58.768957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:58.768957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.769549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:58.769549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:58.769549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:58.769549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:58.769549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:58.769549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:58.769549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:58.769549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:58.769549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.769549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:58.769549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:58.769549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.769549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:58.770118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.770118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:58.771003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:58.772022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.772864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:58.772864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.772864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:58.772864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:58.772864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:58.772864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:58.772864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:45:58.774079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:58.774254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.774710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:58.774710 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.774710 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:58.774710 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:58.774710 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.774710 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:58.774710 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.774710 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:58.775446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:58.775867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:58.775867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:58.775867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.775867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:58.775867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:58.775867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.775867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:58.775867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.775867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:58.776156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.776156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:58.776156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:58.776156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:58.776156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:58.776156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:58.776156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:58.776156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:45:58.776156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:58.776156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:58.776156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.776156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:58.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:58.776877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:58.777286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:58.777782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:58.777782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:58.777782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:58.777782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:58.777782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:58.777782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.777782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:58.777782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:58.777782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.777782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:58.777782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.777782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:58.779040 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:58.779303 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:58.779364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:58.779470 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:58.779706 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:58.779872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.780248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:58.780248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.780248 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:58.780248 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:58.780248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.780248 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:58.780248 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.780248 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:45:58.780767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:45:58.781189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:58.781189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:45:58.781189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.781189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:45:58.781189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:45:58.781189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.781189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:45:58.781189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.781189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:45:58.781476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.781476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:58.781476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:58.781476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:58.781476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:58.781476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:58.781476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:58.781476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:58.781476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:58.781476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.781476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:58.781788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:58.782594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:58.783118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:58.783118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:58.783118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:58.783118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:58.783118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.783118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:58.783118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:58.783118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.783118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:58.783118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.783118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:45:58.790710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:45:58.791067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:45:58.791230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:45:58.791495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:45:58.792672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:58.792925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:45:58.793088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.793852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:45:58.793852 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.793852 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:45:58.793852 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:45:58.793852 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.793852 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:45:58.793852 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.793852 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:45:58.795610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.795610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:45:58.795610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:58.795610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:45:58.795610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:45:58.795610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:58.795610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:58.795610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.795610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:58.795610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:58.795610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:45:58.795610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.796472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:45:58.796472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:58.796472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:45:58.796472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:58.796472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:45:58.796472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:45:58.796472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:58.796472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:45:58.796472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.796472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:45:58.796472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:45:58.796472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.796472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:45:58.797249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.797249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:45:58.797605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:45:58.798118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:45:58.798676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:45:58.798676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:45:58.798676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:01.071924 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:01.072592 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:01.072744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:01.073994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:01.107988 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:01.108604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:01.109604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:01.109604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:01.109604 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:01.109604 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:01.109604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:01.109604 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:01.109604 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:01.109604 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:01.110592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:01.110592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:01.110592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:01.110592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:01.110592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:01.110592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:01.110592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:01.110592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:01.110592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:01.110592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:01.110592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:01.110592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:01.111284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:01.111284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:01.111284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:01.111284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:01.111284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:01.111284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:01.111284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:01.111284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:01.111284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:01.111284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:01.111284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:01.111284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:01.111284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:01.111910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:01.112679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:01.113600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:01.113600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:01.113600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:01.113600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:01.113600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:01.113600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:03.765560 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:03.766063 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:03.766632 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:03.767006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:03.767361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:03.767605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.768513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:03.768513 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.768513 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:03.768513 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:03.768513 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.768513 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:03.768513 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.768513 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:03.769515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.769515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:03.769515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:03.769515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:03.769515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:03.769515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:03.769515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:03.769515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.769515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:03.769515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:03.769515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:03.769515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.770090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:03.770090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:03.770090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:03.770090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:03.770090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:03.770090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:03.770090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:03.770090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:03.770090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.770090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:03.770090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:03.770090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.770090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:03.770827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:03.771570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.772608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:03.772608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:03.772608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.772608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:03.772608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.772608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:03.772608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:03.772608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:03.772608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:03.772608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:46:03.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:03.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.773929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:03.773929 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.773929 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:03.773929 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:03.773929 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.773929 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:03.773929 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.773929 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:03.774685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:03.775426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:03.775426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:03.775426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.775426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:03.775426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:03.775426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.775426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:03.775426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.775426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:03.775821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.775821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:03.775821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:03.775821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:03.775821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:03.775821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:03.775821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:03.775821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:46:03.775821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:03.775821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:03.775821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.775821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:03.776377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:03.776963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:03.777620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:03.778410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:03.778410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:03.778410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:03.778410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:03.778410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:03.778410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.778410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:03.778410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:03.778410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.778410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:03.778410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.778410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:03.779490 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:03.779762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:03.779859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:03.779977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:03.780331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:03.780615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.781053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:03.781053 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.781053 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:03.781053 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:03.781053 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.781053 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:03.781053 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.781053 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:03.781745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:03.782593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:03.783371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:03.783371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:03.783371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:03.783371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:03.783371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.783371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:03.783371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:03.783371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:03.783371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.783371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:03.783371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:03.783371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:03.783371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:03.783942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:03.783942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:03.783942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:03.783942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:03.783942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.783942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:03.783942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:03.783942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.783942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:03.783942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.783942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:03.783942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:03.784474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.785242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:03.791136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:03.791551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:03.791648 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:03.791808 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:03.792304 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:03.792767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:03.793091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.793926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:03.793926 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.793926 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:03.793926 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:03.793926 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.793926 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:03.793926 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.793926 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:03.794829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.794829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:03.794829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:03.794829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:03.794829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:03.794829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:03.794829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:03.794829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.794829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:03.794829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:03.794829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:03.794829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.795240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:03.795240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:03.795240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:03.795240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:03.795240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:03.795240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:03.795240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:03.795240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:03.795240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.795240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:03.795240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:03.795240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.795240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:03.795594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.795594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:03.795818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:03.796267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:03.796736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:03.796736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:03.796736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:06.072614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:06.074742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:06.074742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:06.074742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:06.076466 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=434 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:46:06.109503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:06.110468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:06.110468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:06.110468 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:06.110468 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:06.110468 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:06.110468 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:06.110468 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:06.110468 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:06.110468 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:06.111063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:06.111063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:06.111063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:06.111063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:06.111063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:06.111063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:06.111063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:06.111063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:06.111063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:06.111063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:06.111063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:06.111063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:06.111717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:06.111717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:06.111717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:06.111717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:06.111717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:06.111717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:06.111717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:06.111717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:06.111717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:06.111717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:06.111717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:06.111717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:06.111717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:06.112532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:06.113316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:06.114345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:06.114345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:06.114345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:06.114345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:06.114345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:06.114345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:08.766245 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:08.766901 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:08.767083 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:08.768399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:08.768597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:08.768848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.769746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:08.769746 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.769746 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:08.769746 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:08.769746 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.769746 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:08.769746 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.769746 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:08.771201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.771201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:08.771201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:08.771201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:08.771201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:08.771201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:08.771201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:08.771201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.771201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:08.771201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:08.771201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:08.771201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.771507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:08.771507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:08.771507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:08.771507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:08.771507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:08.771507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:08.771507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:08.771507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:08.771507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.771507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:08.771507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:08.771507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.771507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:08.771802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.771802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:08.773551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:08.774084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.774652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:08.774652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.774652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:08.774652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:08.774652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:08.774652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:08.774652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:46:08.774652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:08.774971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.775554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:08.775554 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.775554 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:08.775554 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:08.775554 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.775554 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:08.775554 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.775554 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:08.776185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:08.776732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:08.776732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:08.776732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.776732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:08.776732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:08.776732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.776732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:08.776732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.776732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:08.777133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.777133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:08.777133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:08.777133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:08.777133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:08.777133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:08.777133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:08.777133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:46:08.777133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:08.777133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:08.777133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.777133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:08.777564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:08.777944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.777944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:08.777944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.777944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:08.778143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:08.778633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:08.779146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.779146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:08.779146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:08.779146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.779146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:08.779146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.779146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:08.779938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:08.780233 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:08.780356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:08.780476 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:08.781027 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:08.781500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.782200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:08.782200 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.782200 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:08.782200 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:08.782200 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.782200 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:08.782200 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.782200 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:08.782971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:08.783276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:08.783276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:08.783276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.783276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:08.783276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:08.783276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.783276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:08.783276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.783276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:08.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:08.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:08.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:08.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:08.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:08.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:08.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:08.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:08.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:08.784266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:08.784543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.784543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:08.784543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.784543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:08.784955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:08.785321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:08.785682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.785682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:08.785682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.785682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:08.791714 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:08.792012 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:08.792165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:08.792430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:08.792968 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:08.793276 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:08.793458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.794199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:08.794199 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.794199 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:08.794199 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:08.794199 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.794199 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:08.794199 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.794199 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:08.795827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.795827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:08.795827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:08.795827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:08.795827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:08.795827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:08.795827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:08.795827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.795827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:08.795827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:08.795827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:08.795827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.796719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:08.796719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:08.796719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:08.796719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:08.796719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:08.796719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:08.796719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:08.796719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:08.796719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.796719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:08.796719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:08.796719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.796719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:08.797725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.797725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:08.798441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:08.799168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:08.800127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:08.800127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:08.800127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:11.073260 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:11.073899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:11.073899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:11.074853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:11.103404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:11.103655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:11.104231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:11.104231 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:11.104231 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:11.104231 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:11.104231 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:11.104231 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:11.104231 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:11.104231 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:11.105142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:11.105142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:11.105142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:11.105142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:11.105142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:11.105142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:11.105142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:11.105142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:11.105142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:11.105142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:11.105142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:11.105142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:11.105817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:11.105817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:11.105817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:11.105817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:11.105817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:11.105817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:11.105817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:11.105817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:11.105817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:11.105817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:11.105817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:11.105817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:11.105817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:11.106429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:11.107193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:11.108084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:11.108084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:11.108084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:11.108084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:11.108084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:11.108084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:13.767878 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:13.768510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:13.768769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:13.769827 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:13.770152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:13.770466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.771339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:13.771339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.771339 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:13.771339 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:13.771339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.771339 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:13.771339 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.771339 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:13.772341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.772341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:13.772341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:13.772341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:13.772341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:13.772341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:13.772341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:13.772341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.772341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:13.772341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:13.772341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:13.772341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.772755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:13.772755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:13.772755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:13.772755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:13.772755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:13.772755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:13.772755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:13.772755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:13.772755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.772755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:13.772755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:13.772755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.772755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:13.773352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:13.774069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.775015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:13.775015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:13.775015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.775015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:13.775015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.775015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:13.775015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:13.775479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:13.775660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:13.776228 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:46:13.777115 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:13.777449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.778119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:13.778119 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.778119 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:13.778119 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:13.778119 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.778119 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:13.778119 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.778119 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:13.778908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:13.779289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:13.779289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:13.779289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.779289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:13.779289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:13.779289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.779289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:13.779289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.779289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:13.779717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.779717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:13.779717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:13.779717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:13.779717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:13.779717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:13.779717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:13.779717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:46:13.779717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:13.779717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:13.779717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.779717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:13.780081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:13.780462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.780462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:13.780462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.780462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:13.782606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:13.783173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:13.783755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.783755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:13.783755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:13.783755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.783755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:13.783755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.783755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:13.783755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:13.783755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:13.783755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:13.784109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:13.784109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:13.784109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.784519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:13.784519 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.784519 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:13.784519 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:13.784519 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.784519 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:13.784519 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.784519 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:13.785511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:13.785999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:13.785999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:13.785999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.785999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:13.785999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:13.785999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.785999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:13.785999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.785999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:13.786665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.786665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:13.786665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:13.786665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:13.786665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:13.786665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:13.786665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:13.786665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:13.786665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:13.786665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.786665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:13.787437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:13.788179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:13.788869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:13.789706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:13.789706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:13.789706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:13.789706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:13.789706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.789706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:13.789706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:13.789706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.789706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:13.789706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.789706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:13.792180 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:13.792475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:13.792542 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:13.792660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:13.792962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:13.793187 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:13.793382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:13.793971 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.793971 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:13.793971 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:13.793971 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.793971 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:13.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.793971 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:13.794572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.794572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:13.794572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:13.794572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:13.794572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:13.794572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:13.794572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:13.794572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.794572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:13.794572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:13.794572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:13.794572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.795178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:13.795178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:13.795178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:13.795178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:13.795178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:13.795178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:13.795178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:13.795178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:13.795178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.795178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:13.795178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:13.795178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.795178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:13.795666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:13.796076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:13.796620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:13.796620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:13.796620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:13.796620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:13.796620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:13.796620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:16.074111 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:16.074861 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:16.074861 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:16.076068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:16.110965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:16.111430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:16.112257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:16.112257 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:16.112257 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:16.112257 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:16.112257 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:16.112257 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:16.112257 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:16.112257 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:16.113004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:16.113004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:16.113004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:16.113004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:16.113004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:16.113004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:16.113004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:16.113004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:16.113004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:16.113004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:16.113004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:16.113004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:16.113735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:16.113735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:16.113735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:16.113735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:16.113735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:16.113735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:16.113735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:16.113735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:16.113735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:16.113735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:16.113735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:16.113735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:16.113735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:16.114380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:16.115108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:16.116027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:16.116027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:16.116027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:16.116027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:16.116027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:16.116027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:18.768552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:18.769401 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:18.769506 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:18.770635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:18.771120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:18.771379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.772265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:18.772265 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.772265 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:18.772265 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:18.772265 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.772265 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:18.772265 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.772265 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:18.773887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.773887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:18.773887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:18.773887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:18.773887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:18.773887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:18.773887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:18.773887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.773887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:18.773887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:18.773887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:18.773887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.774543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:18.774543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:18.774543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:18.774543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:18.774543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:18.774543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:18.774543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:18.774543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:18.774543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.774543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:18.774543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:18.774543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.774543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:18.775419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.775419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:18.776305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:18.777169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.777915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:18.777915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.777915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:18.777915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:18.777915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:18.777915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:18.777915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:46:18.777915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:18.778433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.778990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:18.778990 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.778990 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:18.778990 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:18.778990 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.778990 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:18.778990 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.778990 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:18.779998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:18.780621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:18.780621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:18.780621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.780621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:18.780621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:18.780621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.780621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:18.780621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.780621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:18.781050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.781050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:18.781050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:18.781050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:18.781050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:18.781050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:18.781050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:18.781050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:46:18.781050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:18.781050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:18.781050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.781050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:18.781606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:18.782280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:18.782934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:18.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:18.784645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:18.784645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:18.784645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:18.785058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.785772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:18.785772 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.785772 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:18.785772 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:18.785772 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.785772 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:18.785772 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.785772 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:18.786500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:18.787085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:18.787085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:18.787085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.787085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:18.787085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:18.787085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.787085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:18.787085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.787085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:18.787429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.787429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:18.787429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:18.787429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:18.787429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:18.787429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:18.787429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:18.787429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:18.787429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:18.787429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.787429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:18.787909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:18.788614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:18.789373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:18.790020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:18.790020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:18.790020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:18.790020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:18.790020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.790020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:18.790020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:18.790020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.790020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:18.790020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.790020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:18.792547 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:18.792785 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:18.792896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:18.793087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:18.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:18.793844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:18.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.794799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:18.794799 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.794799 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:18.794799 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:18.794799 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.794799 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:18.794799 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.794799 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:18.795752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.795752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:18.795752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:18.795752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:18.795752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:18.795752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:18.795752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:18.795752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.795752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:18.795752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:18.795752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:18.795752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.796164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:18.796164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:18.796164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:18.796164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:18.796164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:18.796164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:18.796164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:18.796164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:18.796164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.796164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:18.796164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:18.796164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.796164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:18.796521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.796521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:18.796766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:18.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:18.797694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:18.797694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:18.797694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:21.075142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:21.075142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:21.075543 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:21.076479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:21.110654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:21.110889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:21.111433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:21.111433 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:21.111433 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:21.111433 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:21.111433 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:21.111433 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:21.111433 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:21.111433 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:21.112088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:21.112088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:21.112088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:21.112088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:21.112088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:21.112088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:21.112088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:21.112088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:21.112088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:21.112088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:21.112088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:21.112088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:21.112683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:21.112683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:21.112683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:21.112683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:21.112683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:21.112683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:21.112683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:21.112683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:21.112683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:21.112683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:21.112683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:21.112683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:21.112683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:21.113248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:21.113924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:21.114734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:21.114734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:21.114734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:21.114734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:21.114734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:21.114734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:23.769263 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:23.770024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:23.770024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:23.770024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:23.770177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:23.770275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.770788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:23.770788 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.770788 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:23.770788 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:23.770788 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.770788 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:23.770788 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.770788 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:23.771775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.771775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:23.771775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:23.771775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:23.771775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:23.771775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:23.771775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:23.771775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.771775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:23.771775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:23.771775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:23.771775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.772420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:23.772420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:23.772420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:23.772420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:23.772420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:23.772420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:23.772420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:23.772420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:23.772420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.772420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:23.772420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:23.772420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.772420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:23.773407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:23.774318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.775170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:23.775170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:23.775170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.775170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:23.775170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.775170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:23.776839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:23.777102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:23.777238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:23.777356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:46:23.777678 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:23.777917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.778530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:23.778530 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.778530 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:23.778530 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:23.778530 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.778530 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:23.778530 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.778530 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:23.779273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:23.780180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:23.780572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:23.780572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:23.780572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:46:23.780572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:23.780572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:23.780572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.780572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:23.780572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:23.780572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:23.780572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.780572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:23.780572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:23.780572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:23.780885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:23.780885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:23.780885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:23.780885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:23.780885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:23.780885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.780885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:23.780885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:23.780885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.780885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:23.780885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.780885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:23.781173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:23.781578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:23.782002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.782002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:23.782002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:23.782002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.782002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:23.782002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.782002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:23.783784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:23.783984 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:23.784082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:23.784216 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:23.784578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:23.784837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.785288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:23.785288 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.785288 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:23.785288 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:23.785288 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.785288 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:23.785288 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.785288 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:23.785952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:23.786347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:23.786347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:23.786347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.786347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:23.786347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:23.786347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.786347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:23.786347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.786347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:23.786866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.786866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:23.786866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:23.786866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:23.786866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:23.786866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:23.786866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:23.786866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:23.786866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:23.786866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.786866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:23.787458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:23.788310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:23.788656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:23.789113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:23.789113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:23.789113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:23.789113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:23.789113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.789113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:23.789113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:23.789113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.789113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:23.789113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.789113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:23.792926 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:23.793175 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:23.793292 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:23.793433 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:23.793754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:23.794070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:23.794296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.794711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:23.794711 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.794711 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:23.794711 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:23.794711 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.794711 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:23.794711 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.794711 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:23.795522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.795522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:23.795522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:23.795522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:23.795522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:23.795522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:23.795522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:23.795522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.795522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:23.795522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:23.795522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:23.795522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.795857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:23.795857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:23.795857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:23.795857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:23.795857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:23.795857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:23.795857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:23.795857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:23.795857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.795857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:23.795857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:23.795857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.795857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:23.796159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.796159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:23.796516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:23.797391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:23.798191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:23.798191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:23.798191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:26.075172 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:26.076187 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:26.076187 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:26.077090 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:26.111099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:26.111414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:26.112079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:26.112079 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:26.112079 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:26.112079 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:26.112079 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:26.112079 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:26.112079 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:26.112079 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:26.113030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:26.113030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:26.113030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:26.113030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:26.113030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:26.113030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:26.113030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:26.113030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:26.113030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:26.113030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:26.113030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:26.113030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:26.113949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:26.113949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:26.113949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:26.113949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:26.113949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:26.113949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:26.113949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:26.113949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:26.113949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:26.113949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:26.113949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:26.113949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:26.113949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:26.114772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:26.115708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:26.117736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:26.117736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:26.117736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:26.117736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:26.117736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:26.117736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:28.769660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:28.770251 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:28.770251 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:28.772756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:28.772756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:28.772756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.772756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:28.772756 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.772756 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:28.772756 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:28.772756 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.772756 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:28.772756 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.772756 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:28.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:28.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:28.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:28.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:28.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:28.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:28.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:28.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:28.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:28.773096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.773379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:28.773379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:28.773379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:28.773379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:28.773379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:28.773379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:28.773379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:28.773379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:28.773379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.773379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:28.773379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:28.773379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.773379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:28.773640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.773640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:28.773771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:28.774129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.774537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:28.774537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.774537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:28.777297 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:28.777456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:28.777492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:28.777578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:46:28.778009 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:28.778315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.779301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:28.779301 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.779301 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:28.779301 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:28.779301 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.779301 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:28.779301 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.779301 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:28.779688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:28.780450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:28.781113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:28.781113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:28.781113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:46:28.781113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:28.781113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:28.781113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.781113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:28.781113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:28.781113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:28.781113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.781113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:28.781113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:28.781113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:28.781797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:28.781797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:28.781797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:28.781797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:28.781797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:28.781797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.781797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:28.781797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:28.781797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.781797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:28.781797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.781797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:28.784269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:28.785067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:28.785857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.785857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:28.785857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:28.785857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.785857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:28.785857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.785857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:28.785857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:28.785857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:28.785857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:28.786362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:28.786362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:28.786362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.786362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:28.786362 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.786362 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:28.786362 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:28.786362 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.786362 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:28.786362 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.786362 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:28.786853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:28.790326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:28.791058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:28.791058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:28.791058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:28.791058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:28.791058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.791058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:28.791058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:28.791058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:28.791058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.791058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:28.791058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:28.791058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:28.791058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:28.793080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:28.793080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:28.793080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:28.793080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:28.793080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.793080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:28.793080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:28.793080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.793080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:28.793080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.793080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:28.793080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:28.793837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.794648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:28.795453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:28.795453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:28.795453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:28.795453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:28.795453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:28.795453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:28.795738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.795738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:28.795738 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.795738 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:28.795738 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:28.795738 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.795738 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:28.795738 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.795738 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:28.796418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.796418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:28.796418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:28.796418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:28.796418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:28.796418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:28.796418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:28.796418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.796418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:28.796418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:28.796418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:28.796418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.797080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:28.797080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:28.797080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:28.797080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:28.797080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:28.797080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:28.797080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:28.797080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:28.797080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.797080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:28.797080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:28.797080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.797080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:28.797610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:28.798257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:28.799029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:28.799029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:28.799029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:28.799029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:28.799029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:28.799029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:31.075862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:31.076487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:31.076654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:31.077820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:31.112397 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:31.112397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:31.112397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:31.112397 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:31.112397 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:31.112397 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:31.112397 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:31.112397 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:31.112397 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:31.112397 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:31.113372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:31.113372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:31.113372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:31.113372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:31.113372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:31.113372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:31.113372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:31.113372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:31.113372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:31.113372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:31.113372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:31.113372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:31.113732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:31.113732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:31.113732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:31.113732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:31.113732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:31.113732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:31.113732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:31.113732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:31.113732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:31.113732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:31.113732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:31.113732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:31.113732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:31.114063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:31.114063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:31.114472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:31.114985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:31.115558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:31.115558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:31.115558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:33.770170 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:33.771073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:33.771073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:33.772464 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:33.772464 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:33.772464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.773401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:33.773401 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.773401 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:33.773401 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:33.773401 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.773401 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:33.773401 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.773401 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:33.774884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.774884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:33.774884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:33.774884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:33.774884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:33.774884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:33.774884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:33.774884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.774884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:33.774884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:33.774884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:33.774884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.775551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:33.775551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:33.775551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:33.775551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:33.775551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:33.775551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:33.775551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:33.775551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:33.775551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.775551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:33.775551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:33.775551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.775551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:33.776062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.776062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:33.777605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:33.779164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.780020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:33.780020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.780020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:33.780020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:33.780020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:33.780020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:33.780020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:46:33.780020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:33.780249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.780249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:33.780249 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.780249 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:33.780249 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:33.780249 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.780249 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:33.780249 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.780249 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:33.780595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:33.781203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:33.781203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:33.781203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.781203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:33.781203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:33.781203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.781203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:33.781203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.781203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:33.781595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.781595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:33.781595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:33.781595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:33.781595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:33.781595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:33.781595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:33.781595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:46:33.781595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:33.781595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:33.781595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.781595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:33.781854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:33.782101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.782101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:33.782101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.782101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:33.782925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:33.783245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:33.783580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.783580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:33.783580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:33.783580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.783580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:33.783580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.783580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:33.784452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:33.784752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:33.784890 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:33.785049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:33.785504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:33.785844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.786664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:33.786664 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.786664 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:33.786664 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:33.786664 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.786664 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:33.786664 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.786664 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:33.787545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:33.788007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:33.788007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:33.788007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.788007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:33.788007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:33.788007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.788007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:33.788007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.788007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:33.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:33.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:33.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:33.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:33.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:33.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:33.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:33.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:33.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:33.789194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:33.789630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.789630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:33.789630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.789630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:33.790167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:33.790720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:33.791279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.791279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:33.791279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.791279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:33.793777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:33.794145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:33.794324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:33.794538 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:33.794980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:33.795323 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:33.795536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.796257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:33.796257 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.796257 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:33.796257 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:33.796257 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.796257 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:33.796257 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.796257 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:33.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:33.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:33.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:33.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:33.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:33.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:33.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:33.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:33.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:33.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.797723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:33.797723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:33.797723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:33.797723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:33.797723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:33.797723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:33.797723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:33.797723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:33.797723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.797723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:33.797723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:33.797723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.797723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:33.798147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.798147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:33.798647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:33.799674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:33.800330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:33.800330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:33.800330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:36.076561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:36.077413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:36.077413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:36.077929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:36.112687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:36.113089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:36.113977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:36.113977 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:36.113977 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:36.113977 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:36.113977 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:36.113977 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:36.113977 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:36.113977 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:36.115089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:36.115089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:36.115089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:36.115089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:36.115089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:36.115089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:36.115089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:36.115089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:36.115089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:36.115089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:36.115089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:36.115089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:36.115730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:36.115730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:36.115730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:36.115730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:36.115730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:36.115730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:36.115730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:36.115730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:36.115730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:36.115730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:36.115730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:36.115730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:36.115730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:36.116280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:36.116280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:36.116383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:36.117058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:36.117895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:36.117895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:36.117895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:38.770664 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:38.771357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:38.771453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:38.772408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:38.772648 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:38.772771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.773377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:38.773377 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.773377 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:38.773377 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:38.773377 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.773377 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:38.773377 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.773377 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:38.774869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.774869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:38.774869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:38.774869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:38.774869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:38.774869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:38.774869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:38.774869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.774869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:38.774869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:38.774869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:38.774869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.775544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:38.775544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:38.775544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:38.775544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:38.775544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:38.775544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:38.775544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:38.775544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:38.775544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.775544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:38.775544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:38.775544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.775544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:38.776267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.776267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:38.776455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:38.776985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.777626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:38.777626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.777626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:38.777819 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:38.778043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:38.778152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:38.778289 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:46:38.778605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:38.778799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.779298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:38.779298 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.779298 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:38.779298 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:38.779298 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.779298 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:38.779298 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.779298 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:38.780056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:38.780558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:38.780558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:38.780558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.780558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:38.780558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:38.780558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.780558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:38.780558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.780558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:38.780904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.780904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:38.780904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:38.780904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:38.780904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:38.780904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:38.780904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:38.780904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:46:38.780904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:38.780904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:38.780904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.780904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:38.781296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:38.781691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.781691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:38.781691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.781691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:38.781925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:38.782396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:38.782897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.782897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:38.782897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:38.782897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.782897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:38.782897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.782897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:38.784834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:38.785057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:38.785167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:38.785310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:38.786150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:38.786401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.786880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:38.786880 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.786880 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:38.786880 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:38.786880 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.786880 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:38.786880 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.786880 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:38.787621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:38.788077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:38.788077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:38.788077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.788077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:38.788077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:38.788077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.788077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:38.788077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.788077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:38.788630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.788630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:38.788630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:38.788630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:38.788630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:38.788630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:38.788630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:38.788630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:38.788630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:38.788630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.788630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:38.788976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:38.789421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.789421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:38.789421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.789421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:38.789603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:38.790132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:38.790679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.790679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:38.790679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.790679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:38.794234 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:38.794529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:38.794637 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:38.794783 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:38.795095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:38.795339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:38.795482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.795971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:38.795971 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.795971 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:38.795971 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:38.795971 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.795971 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:38.795971 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.795971 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:38.796807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.796807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:38.796807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:38.796807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:38.796807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:38.796807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:38.796807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:38.796807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.796807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:38.796807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:38.796807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:38.796807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.797830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:38.797830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:38.797830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:38.797830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:38.797830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:38.797830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:38.797830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:38.797830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:38.797830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.797830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:38.797830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:38.797830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.797830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:38.798231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:38.798667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:38.799234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:38.799234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:38.799234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:38.799234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:38.799234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:38.799234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:41.077444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:41.078162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:41.078162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:41.079111 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:41.113770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:41.113770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:41.114019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:41.114019 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:41.114019 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:41.114019 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:41.114019 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:41.114019 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:41.114019 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:41.114019 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:41.114569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:41.114569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:41.114569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:41.114569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:41.114569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:41.114569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:41.114569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:41.114569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:41.114569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:41.114569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:41.114569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:41.114569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:41.115439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:41.115439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:41.115439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:41.115439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:41.115439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:41.115439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:41.115439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:41.115439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:41.115439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:41.115439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:41.115439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:41.115439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:41.115439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:41.116131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:41.116715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:41.117268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:41.117268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:41.117268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:41.117268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:41.117268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:41.117268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:43.771199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:43.771802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:43.771802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:43.772561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:43.772561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:43.772689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.773156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:43.773156 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.773156 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:43.773156 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:43.773156 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.773156 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:43.773156 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.773156 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:43.774289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.774289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:43.774289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:43.774289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:43.774289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:43.774289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:43.774289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:43.774289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.774289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:43.774289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:43.774289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:43.774289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.774762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:43.774762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:43.774762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:43.774762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:43.774762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:43.774762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:43.774762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:43.774762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:43.774762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.774762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:43.774762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:43.774762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.774762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:43.776067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:43.776530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.776989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:43.776989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:43.776989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.776989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:43.776989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.776989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:43.778194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:43.778526 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:43.778674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:43.778838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:46:43.779146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:43.779356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.779807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:43.779807 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.779807 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:43.779807 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:43.779807 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.779807 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:43.779807 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.779807 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:43.780726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:43.781323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:43.781323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:43.781323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.781323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:43.781323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:43.781323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.781323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:43.781323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.781323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:43.782015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.782015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:43.782015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:43.782015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:43.782015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:43.782015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:43.782015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:43.782015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:46:43.782015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:43.782015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:43.782015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.782015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:43.783364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:43.784124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:43.784521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:43.784964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:43.784964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:43.784964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:43.784964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:43.784964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:43.784964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.784964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:43.784964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:43.784964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.784964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:43.784964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.784964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:43.786329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:43.786329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:43.786329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:43.786329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:43.786329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:43.787567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.787567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:43.787567 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.787567 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:43.787567 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:43.787567 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.787567 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:43.787567 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.787567 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:43.788344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:43.789052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:43.789052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:43.789052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.789052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:43.789052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:43.789052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.789052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:43.789052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.789052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:43.789678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.789678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:43.789678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:43.789678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:43.789678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:43.789678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:43.789678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:43.789678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:43.789678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:43.789678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.789678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:43.790631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:43.791250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:43.791678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:43.791999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:43.791999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:43.791999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:43.791999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:43.791999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.791999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:43.791999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:43.791999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.791999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:43.791999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.791999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:43.794753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:43.794975 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:43.795074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:43.795227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:43.795620 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:43.795938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:43.796142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.796747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:43.796747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.796747 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:43.796747 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:43.796747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.796747 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:43.796747 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.796747 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:43.797463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.797463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:43.797463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:43.797463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:43.797463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:43.797463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:43.797463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:43.797463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.797463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:43.797463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:43.797463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:43.797463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:43.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:43.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:43.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:43.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:43.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:43.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:43.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:43.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:43.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:43.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.797806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:43.798130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.798130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:43.798593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:43.799554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:43.800506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:43.800506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:43.800506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:46.077946 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:46.078733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:46.078802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:46.080058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:46.120022 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:46.120265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:46.120972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:46.120972 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:46.120972 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:46.120972 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:46.120972 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:46.120972 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:46.120972 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:46.120972 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:46.121998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:46.121998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:46.121998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:46.121998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:46.121998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:46.121998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:46.121998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:46.121998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:46.121998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:46.121998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:46.121998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:46.121998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:46.123001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:46.123001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:46.123001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:46.123001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:46.123001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:46.123001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:46.123001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:46.123001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:46.123001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:46.123001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:46.123001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:46.123001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:46.123001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:46.123891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:46.124740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:46.125940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:46.125940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:46.125940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:46.125940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:46.125940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:46.125940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:48.772050 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:48.772994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:48.772994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:48.773687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:48.773994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:48.774249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.775018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:48.775018 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.775018 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:48.775018 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:48.775018 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.775018 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:48.775018 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.775018 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:48.776074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.776074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:48.776074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:48.776074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:48.776074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:48.776074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:48.776074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:48.776074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.776074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:48.776074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:48.776074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:48.776074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.776770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:48.776770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:48.776770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:48.776770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:48.776770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:48.776770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:48.776770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:48.776770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:48.776770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.776770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:48.776770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:48.776770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.776770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:48.777426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:48.778263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.779316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:48.779316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:48.779316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.779316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:48.779316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.779316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:48.779316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:48.779316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:48.779316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:48.779316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:46:48.779997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:48.779997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.782405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:48.783009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.783396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:48.783396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:48.783396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:48.783396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:48.783396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:48.783396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:48.783396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:46:48.783396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:48.783396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:48.783396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.783396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:48.783396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:48.783396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.783657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:48.783907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.783907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:48.784093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:48.784431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:48.784852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.784852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:48.784852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:48.784852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.784852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:48.784852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.784852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:48.785413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:48.785684 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:48.785839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:48.786023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:48.786506 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:48.786843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.787501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:48.787501 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.787501 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:48.787501 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:48.787501 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.787501 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:48.787501 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.787501 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:48.788280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:48.788898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:48.788898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:48.788898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.788898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:48.788898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:48.788898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.788898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:48.788898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.788898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:48.789271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.789271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:48.789271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:48.789271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:48.789271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:48.789271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:48.789271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:48.789271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:48.789271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:48.789271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.789271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:48.789700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:48.790346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:48.790926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:48.791747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:48.791747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:48.791747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:48.791747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:48.791747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.791747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:48.791747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:48.791747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.791747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:48.791747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.791747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:48.795158 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:48.795440 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:48.795550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:48.795704 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:48.796123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:48.796557 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:48.796767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.797491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:48.797491 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.797491 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:48.797491 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:48.797491 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.797491 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:48.797491 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.797491 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:48.798329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.798329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:48.798329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:48.798329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:48.798329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:48.798329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:48.798329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:48.798329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.798329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:48.798329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:48.798329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:48.798329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:48.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:48.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:48.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:48.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:48.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:48.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:48.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:48.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:48.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:48.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:48.799436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:48.800091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:48.800800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:48.800800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:48.800800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:48.800800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:48.800800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:48.800800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:51.078590 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:51.079461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:51.079461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:51.080580 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:51.118777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:51.119162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:51.119996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:51.119996 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:51.119996 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:51.119996 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:51.119996 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:51.119996 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:51.119996 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:51.119996 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:51.120827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:51.120827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:51.120827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:51.120827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:51.120827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:51.120827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:51.120827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:51.120827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:51.120827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:51.120827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:51.120827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:51.120827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:51.121406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:51.121406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:51.121406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:51.121406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:51.121406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:51.121406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:51.121406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:51.121406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:51.121406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:51.121406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:51.121406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:51.121406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:51.121406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:51.122055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:51.122934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:51.123816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:51.123816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:51.123816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:51.123816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:51.123816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:51.123816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:53.772606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:53.773301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:53.773440 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:53.774526 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:53.774866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:53.775096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.776033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:53.776033 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.776033 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:53.776033 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:53.776033 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.776033 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:53.776033 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.776033 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:53.777567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.777567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:53.777567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:53.777567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:53.777567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:53.777567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:53.777567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:53.777567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.777567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:53.777567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:53.777567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:53.777567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.777967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:53.777967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:53.777967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:53.777967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:53.777967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:53.777967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:53.777967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:53.777967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:53.777967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.777967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:53.777967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:53.777967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.777967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:53.778362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.778362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:53.779200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:53.779708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.780416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:53.780416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.780416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:53.780416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:53.780416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:53.780416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:53.780593 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:46:53.781031 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:53.781338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.783065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:53.783065 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.783065 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:53.783065 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:53.783065 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.783065 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:53.783065 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.783065 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:53.784224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:53.784669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:53.784669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:53.784669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.784669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:53.784669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:53.784669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.784669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:53.784669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.784669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:53.785590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.785590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:53.785590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:53.785590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:53.785590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:53.785590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:53.785590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:53.785590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:46:53.785590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:53.785590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:53.785590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.785590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:53.785945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:53.786313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.786313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:53.786313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.786313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:53.786313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:53.786675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:53.787110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:53.787599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.787599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:53.787599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:53.787599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.787599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:53.787599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.787599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:53.787599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:53.787599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:53.787599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:53.787892 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:53.787892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.787979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:53.787979 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.787979 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:53.787979 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:53.787979 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.787979 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:53.787979 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.787979 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:53.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:53.788933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:53.788933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:53.788933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.788933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:53.788933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:53.788933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.788933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:53.788933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.788933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:53.789144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.789144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:53.789144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:53.789144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:53.789144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:53.789144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:53.789144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:53.789144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:53.789144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:53.789144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.789144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:53.789451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:53.789764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.789764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:53.789764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.789764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:53.789863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:53.790289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:53.790719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.790719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:53.790719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.790719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:53.795587 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:53.795776 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:53.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:53.796019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:53.796353 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:53.796579 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:53.796724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:53.797224 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.797224 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:53.797224 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:53.797224 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.797224 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:53.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.797224 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:53.798000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.798000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:53.798000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:53.798000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:53.798000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:53.798000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:53.798000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:53.798000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.798000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:53.798000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:53.798000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:53.798000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:53.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:53.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:53.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:53.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:53.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:53.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:53.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:53.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:53.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:53.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:53.798684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.798684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:53.798922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:53.799368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:53.799867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:53.799867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:53.799867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:56.079361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:56.080195 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:56.080298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:56.081865 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:56.118831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:56.119270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:56.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:56.120168 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:56.120168 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:56.120168 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:56.120168 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:56.120168 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:56.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:56.120168 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:56.120927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:56.120927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:56.120927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:56.120927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:56.120927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:56.120927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:56.120927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:56.120927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:56.120927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:56.120927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:56.120927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:56.120927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:56.121526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:56.121526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:56.121526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:56.121526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:56.121526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:56.121526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:56.121526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:56.121526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:56.121526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:56.121526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:56.121526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:56.121526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:56.121526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:56.122095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:56.122768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:56.123618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:56.123618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:56.123618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:56.123618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:56.123618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:56.123618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:58.773259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:58.773824 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:58.774011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:58.775089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:58.775352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:58.775515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.776190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:58.776190 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.776190 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:58.776190 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:58.776190 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.776190 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:58.776190 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.776190 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:58.777702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.777702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:58.777702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:58.777702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:58.777702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:58.777702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:58.777702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:58.777702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.777702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:58.777702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:58.777702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:58.777702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.778010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:58.778010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:58.778010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:58.778010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:58.778010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:58.778010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:58.778010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:58.778010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:58.778010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.778010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:58.778010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:58.778010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.778010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:58.778310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.778310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:58.779932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:58.780644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.781410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:58.781410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.781410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:58.781410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:58.781410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:58.781410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:58.781410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:46:58.781920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:58.782172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.782849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:58.782849 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.782849 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:58.782849 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:58.782849 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.782849 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:58.782849 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.782849 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:58.783326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:58.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:58.784860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:58.784860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:58.784860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:46:58.784860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:58.784860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:58.784860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.784860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:58.784860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:58.784860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:58.784860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.784860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:58.784860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:58.784860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:58.785467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:58.785467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:58.785467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:58.785467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:58.785467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:58.785467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.785467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:58.785467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:58.785467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.785467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:58.785467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.785467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:58.786047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:58.787132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:58.788855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.788855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:58.788855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:58.788855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.788855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:58.788855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.788855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:58.788855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:58.788855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:58.788855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:58.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:58.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:58.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:58.789512 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.789512 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:58.789512 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:58.789512 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.789512 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:58.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:58.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:46:58.790065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.790830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:46:58.790830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:46:58.790830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.790830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:46:58.790830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.790830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:46:58.790830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.790830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:58.790830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:58.790830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:58.790830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:58.790830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:58.790830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:58.793364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:58.793364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:58.793364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.793364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:58.793364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:58.793364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:58.793364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.793364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:58.793364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:58.793364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:58.793364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:58.793364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:58.793364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:58.793972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:58.793972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:58.793972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.793972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:58.793972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:58.793972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.793972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:58.793972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.793972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:58.793972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.793972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:58.793972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.794615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.795226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:46:58.795999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:46:58.796304 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:46:58.796427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:46:58.796595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:46:58.797107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:58.797533 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:46:58.797693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.798465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:46:58.798465 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.798465 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:46:58.798465 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:46:58.798465 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.798465 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:46:58.798465 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.798465 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:46:58.799420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.799420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:46:58.799420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:58.799420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:46:58.799420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:46:58.799420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:58.799420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:58.799420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.799420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:58.799420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:58.799420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:46:58.799420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:46:58.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:58.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:46:58.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:58.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:46:58.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:46:58.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:58.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:46:58.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:46:58.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:46:58.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.799967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:46:58.800242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.800242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:46:58.800407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:46:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:46:58.801090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:46:58.801090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:46:58.801090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:01.080043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:01.080678 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:01.080889 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:01.082142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:01.116781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:01.117152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:01.117899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:01.117899 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:01.117899 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:01.117899 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:01.117899 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:01.117899 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:01.117899 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:01.117899 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:01.118833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:01.118833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:01.118833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:01.118833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:01.118833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:01.118833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:01.118833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:01.118833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:01.118833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:01.118833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:01.118833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:01.118833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:01.119257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:01.119257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:01.119257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:01.119257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:01.119257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:01.119257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:01.119257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:01.119257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:01.119257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:01.119257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:01.119257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:01.119257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:01.119257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:01.119674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:01.119674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:01.119923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:01.120306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:01.120660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:01.120660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:01.120660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:03.773944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:03.774973 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:03.774973 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:03.776062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:03.776330 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:03.776491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.777104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:03.777104 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.777104 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:03.777104 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:03.777104 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.777104 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:03.777104 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.777104 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:03.778235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.778235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:03.778235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:03.778235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:03.778235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:03.778235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:03.778235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:03.778235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.778235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:03.778235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:03.778235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:03.778235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.778825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:03.778825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:03.778825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:03.778825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:03.778825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:03.778825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:03.778825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:03.778825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:03.778825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.778825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:03.778825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:03.778825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.778825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:03.779556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.779556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:03.779748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:03.780401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.781095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:03.781095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.781095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:03.781095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:03.781354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:03.781354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:03.781483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:47:03.781954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:03.782295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.782973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:03.782973 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.782973 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:03.782973 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:03.782973 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.782973 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:03.782973 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.782973 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:03.783896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:03.784655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:03.784655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:03.784655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.784655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:03.784655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:03.784655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.784655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:03.784655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.784655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:03.785109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.785109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:03.785109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:03.785109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:03.785109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:03.785109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:03.785109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:03.785109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:47:03.785109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:03.785109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:03.785109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.785109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:03.785653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:03.786297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.786297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:03.786297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.786297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:03.786480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:03.786969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:03.787521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.787521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:03.787521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:03.787521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.787521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:03.787521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.787521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:03.787521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:03.787902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:03.787902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:03.788085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:03.788574 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:03.788916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.789670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:03.789670 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.789670 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:03.789670 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:03.789670 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.789670 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:03.789670 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.789670 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:03.790726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:03.791381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:03.791381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:03.791381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.791381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:03.791381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:03.791381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.791381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:03.791381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.791381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:03.791821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.791821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:03.791821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:03.791821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:03.791821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:03.791821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:03.791821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:03.791821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:03.791821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:03.791821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.791821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:03.792363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:03.792929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.792929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:03.792929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.792929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:03.793408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:03.793941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:03.794463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.794463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:03.794463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.794463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:03.796461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:03.796744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:03.796890 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:03.797093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:03.797545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:03.797880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:03.798094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.798799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:03.798799 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.798799 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:03.798799 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:03.798799 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.798799 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:03.798799 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.798799 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:03.799746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.799746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:03.799746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:03.799746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:03.799746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:03.799746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:03.799746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:03.799746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.799746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:03.799746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:03.799746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:03.799746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.800290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:03.800290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:03.800290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:03.800290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:03.800290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:03.800290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:03.800290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:03.800290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:03.800290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.800290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:03.800290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:03.800290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.800290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:03.800862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.800862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:03.801105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:03.801615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:03.802122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:03.802122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:03.802122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:06.081048 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:06.081830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:06.081830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:06.082724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:06.084467 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:47:06.116557 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:06.116942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:06.117772 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:06.118541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:06.118541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:06.118541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:06.118541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:06.118541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:06.118541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:06.118541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:06.118541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:06.118541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:06.118541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:06.118541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:06.118541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:06.119174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:06.119174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:06.119174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:06.119174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:06.119174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:06.119174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:06.119174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:06.119174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:06.119174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:06.119174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:06.119174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:06.119174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:06.119174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:06.119770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:06.120518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:06.121365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:06.121365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:06.121365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:06.121365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:06.121365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:06.121365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:08.774619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:08.775271 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:08.775271 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:08.776005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:08.776381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:08.776620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.777430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:08.777430 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.777430 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:08.777430 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:08.777430 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.777430 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:08.777430 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.777430 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:08.778431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.778431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:08.778431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:08.778431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:08.778431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:08.778431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:08.778431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:08.778431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.778431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:08.778431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:08.778431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:08.778431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.778809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:08.778809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:08.778809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:08.778809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:08.778809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:08.778809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:08.778809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:08.778809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:08.778809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.778809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:08.778809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:08.778809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.778809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:08.779182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.779182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:08.779531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:08.781081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.782147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:08.782147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.782147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:08.782147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:08.782147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:08.782147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:08.782147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:47:08.782647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:08.782647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.783719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:08.783719 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.783719 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:08.783719 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:08.783719 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.783719 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:08.783719 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.783719 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:08.784703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:08.785920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:08.786512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:08.786512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:08.786512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:47:08.786512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:08.786512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:08.786512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.786512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:08.786512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:08.786512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:08.786512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.786512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:08.786512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:08.786512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:08.786913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:08.786913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:08.786913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:08.786913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:08.786913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:08.786913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.786913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:08.786913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:08.786913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.786913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:08.786913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.786913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:08.787361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:08.787800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:08.788619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.788619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:08.788619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:08.788619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.788619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:08.788619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.788619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:08.788619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:08.788619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:08.788619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:08.789189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:08.789189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:08.789189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.789777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:08.789777 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.789777 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:08.789777 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:08.789777 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.789777 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:08.789777 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.789777 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:08.790550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:08.791458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:08.792086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:08.792086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:08.792086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:08.792086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:08.792086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.792086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:08.792086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:08.792086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:08.792086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.792086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:08.792086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:08.792086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:08.792086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:08.792685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:08.792685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:08.792685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:08.792685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:08.792685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.792685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:08.792685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:08.792685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.792685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:08.792685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.792685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:08.792685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:08.793374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.793761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:08.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:08.797191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:08.797370 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:08.797545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:08.797900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:08.798277 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:08.798473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.799148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:08.799148 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.799148 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:08.799148 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:08.799148 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.799148 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:08.799148 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.799148 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:08.800273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.800273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:08.800273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:08.800273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:08.800273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:08.800273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:08.800273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:08.800273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.800273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:08.800273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:08.800273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:08.800273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.800610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:08.800610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:08.800610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:08.800610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:08.800610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:08.800610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:08.800610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:08.800610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:08.800610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.800610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:08.800610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:08.800610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.800610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:08.800901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.800901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:08.801576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:08.801923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:08.802342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:08.802342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:08.802342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:11.082838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:11.083812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:11.083812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:11.085076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:11.117838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:11.118201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:11.119110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:11.119110 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:11.119110 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:11.119110 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:11.119110 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:11.119110 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:11.119110 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:11.119110 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:11.120169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:11.120169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:11.120169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:11.120169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:11.120169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:11.120169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:11.120169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:11.120169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:11.120169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:11.120169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:11.120169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:11.120169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:11.120934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:11.120934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:11.120934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:11.120934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:11.120934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:11.120934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:11.120934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:11.120934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:11.120934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:11.120934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:11.120934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:11.120934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:11.120934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:11.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:11.122658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:11.123734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:11.123734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:11.123734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:11.123734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:11.123734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:11.123734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:13.775290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:13.775935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:13.776132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:13.777189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:13.777529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:13.777762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.778614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:13.778614 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.778614 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:13.778614 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:13.778614 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.778614 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:13.778614 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.778614 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:13.780129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.780129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:13.780129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:13.780129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:13.780129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:13.780129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:13.780129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:13.780129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.780129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:13.780129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:13.780129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:13.780129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.780600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:13.780600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:13.780600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:13.780600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:13.780600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:13.780600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:13.780600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:13.780600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:13.780600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.780600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:13.780600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:13.780600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.780600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:13.780997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.780997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:13.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:13.782791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.783393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:13.783393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.783393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:13.783393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:13.783393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:13.783393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:13.783393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:47:13.783922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:13.783922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.783922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:13.783922 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.783922 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:13.783922 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:13.783922 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.783922 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:13.783922 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.783922 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:13.786130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:13.786677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:13.786677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:13.786677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.786677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:13.786677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:13.786677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.786677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:13.786677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.786677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:13.786983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.786983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:13.786983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:13.786983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:13.786983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:13.786983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:13.786983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:13.786983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:47:13.786983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:13.786983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:13.786983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.786983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:13.787369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:13.787762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.787762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:13.787762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.787762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:13.787885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:13.788375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:13.788906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.788906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:13.788906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:13.788906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.788906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:13.788906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.788906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:13.788906 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:13.788906 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:13.788906 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:13.789239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:13.789239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:13.789403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.790071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:13.790071 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.790071 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:13.790071 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:13.790071 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.790071 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:13.790071 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.790071 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:13.790819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:13.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:13.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:13.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:13.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:13.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:13.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:13.791813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.791813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:13.791813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:13.791813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:13.791813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:13.791813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:13.791813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:13.791813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:13.791813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:13.791813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.791813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:13.792187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:13.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:13.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.792625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:13.792912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:13.793500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:13.794009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.794009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:13.794009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.794009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:13.797232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:13.797512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:13.797619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:13.797768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:13.798320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:13.798836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:13.798989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.799824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:13.799824 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.799824 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:13.799824 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:13.799824 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.799824 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:13.799824 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.799824 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:13.800649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.800649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:13.800649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:13.800649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:13.800649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:13.800649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:13.800649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:13.800649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.800649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:13.800649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:13.800649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:13.800649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.801061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:13.801061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:13.801061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:13.801061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:13.801061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:13.801061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:13.801061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:13.801061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:13.801061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.801061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:13.801061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:13.801061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.801061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:13.801434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.801434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:13.801632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:13.802112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:13.802675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:13.802675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:13.802675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:16.083740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:16.084584 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:16.084782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:16.085614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:16.118299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:16.118813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:16.119742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:16.119742 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:16.119742 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:16.119742 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:16.119742 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:16.119742 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:16.119742 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:16.119742 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:16.120975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:16.120975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:16.120975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:16.120975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:16.120975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:16.120975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:16.120975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:16.120975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:16.120975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:16.120975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:16.120975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:16.120975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:16.121612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:16.121612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:16.121612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:16.121612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:16.121612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:16.121612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:16.121612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:16.121612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:16.121612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:16.121612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:16.121612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:16.121612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:16.121612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:16.122197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:16.122848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:16.123892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:16.123892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:16.123892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:16.123892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:16.123892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:16.123892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:18.776306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:18.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:18.777374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:18.777872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:18.778049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:18.778167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.778857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:18.778857 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.778857 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:18.778857 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:18.778857 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.778857 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:18.778857 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.778857 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:18.780366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.780366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:18.780366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:18.780366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:18.780366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:18.780366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:18.780366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:18.780366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.780366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:18.780366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:18.780366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:18.780366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:18.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:18.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:18.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:18.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:18.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:18.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:18.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:18.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:18.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:18.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.780943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:18.781412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.781412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:18.781948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:18.782490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.783084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:18.783084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.783084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:18.783367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:18.783655 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:18.783762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:18.783885 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:47:18.784188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:18.784353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.784781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:18.784781 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.784781 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:18.784781 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:18.784781 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.784781 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:18.784781 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.784781 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:18.785558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:18.786114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:18.786114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:18.786114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.786114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:18.786114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:18.786114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.786114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:18.786114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.786114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:18.786478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.786478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:18.786478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:18.786478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:18.786478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:18.786478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:18.786478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:18.786478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:47:18.786478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:18.786478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:18.786478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.786478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:18.786825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:18.787197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.787197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:18.787197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.787197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:18.787401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:18.787832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:18.788373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.788373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:18.788373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:18.788373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.788373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:18.788373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.788373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:18.788606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:18.788750 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:18.788849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:18.788978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:18.789314 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:18.789554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.789997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:18.789997 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.789997 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:18.789997 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:18.789997 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.789997 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:18.789997 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.789997 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:18.790736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:18.791255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:18.791255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:18.791255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.791255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:18.791255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:18.791255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.791255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:18.791255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.791255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:18.791612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.791612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:18.791612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:18.791612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:18.791612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:18.791612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:18.791612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:18.791612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:18.791612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:18.791612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.791612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:18.791944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:18.792341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.792341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:18.792341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.792341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:18.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:18.793564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:18.794071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.794071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:18.794071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.794071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:18.797694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:18.797947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:18.798053 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:18.798232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:18.798493 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:18.798745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:18.798904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:18.801287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:18.802330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:18.803038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:18.803852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:18.805278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:18.805278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:18.805278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:18.805278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:18.805278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:18.805278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:18.805278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:18.805278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:18.805278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:18.805278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:21.084822 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:21.085606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:21.085979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:21.087338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:21.119270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:21.119513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:21.119972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:21.119972 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:21.119972 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:21.119972 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:21.119972 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:21.119972 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:21.119972 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:21.119972 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:21.120923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:21.120923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:21.120923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:21.120923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:21.120923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:21.120923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:21.120923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:21.120923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:21.120923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:21.120923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:21.120923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:21.120923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:21.121514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:21.121514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:21.121514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:21.121514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:21.121514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:21.121514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:21.121514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:21.121514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:21.121514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:21.121514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:21.121514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:21.121514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:21.121514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:21.122069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:21.122746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:21.123516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:21.123516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:21.123516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:21.123516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:21.123516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:21.123516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:23.777028 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:23.777728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:23.777728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:23.778492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:23.778733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:23.779002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.779926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:23.779926 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.779926 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:23.779926 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:23.779926 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.779926 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:23.779926 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.779926 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:23.781424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.781424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:23.781424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:23.781424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:23.781424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:23.781424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:23.781424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:23.781424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.781424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:23.781424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:23.781424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:23.781424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.781871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:23.781871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:23.781871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:23.781871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:23.781871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:23.781871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:23.781871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:23.781871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:23.781871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.781871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:23.781871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:23.781871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.781871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:23.782355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.782355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:23.783024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:23.783523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:23.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:23.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:23.784060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:23.784240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:23.784281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:47:23.784739 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:23.785023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.785747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:23.785747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.785747 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:23.785747 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:23.785747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.785747 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:23.785747 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.785747 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:23.786540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:23.786966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:23.786966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:23.786966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.786966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:23.786966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:23.786966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.786966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:23.786966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.786966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:23.787751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.787751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:23.787751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:23.787751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:23.787751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:23.787751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:23.787751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:23.787751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:47:23.787751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:23.787751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:23.787751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.787751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:23.788149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:23.788564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.788564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:23.788564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.788564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:23.788840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:23.789383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:23.789857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.789857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:23.789857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:23.789857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.789857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:23.789857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.789857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:23.789857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:23.789857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:23.789857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:23.790134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:23.790255 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:23.790530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.791020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:23.791020 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.791020 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:23.791020 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:23.791020 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.791020 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:23.791020 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.791020 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:23.791785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:23.792191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:23.792191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:23.792191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.792191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:23.792191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:23.792191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.792191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:23.792191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.792191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:23.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:23.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:23.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:23.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:23.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:23.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:23.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:23.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:23.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.792679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:23.792985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:23.793370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.793370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:23.793370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.793370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:23.793773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:23.794335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:23.794871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.794871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:23.794871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.794871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:23.798039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:23.798197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:23.798290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:23.798397 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:23.799018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:23.799620 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:23.799620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.800219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:23.800219 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.800219 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:23.800219 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:23.800219 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.800219 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:23.800219 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.800219 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:23.800927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.800927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:23.800927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:23.800927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:23.800927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:23.800927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:23.800927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:23.800927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.800927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:23.800927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:23.800927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:23.800927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.801320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:23.801320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:23.801320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:23.801320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:23.801320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:23.801320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:23.801320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:23.801320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:23.801320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.801320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:23.801320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:23.801320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.801320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:23.801741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:23.802188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:23.802819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:23.802819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:23.802819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:23.802819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:23.802819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:23.802819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:26.085570 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:26.086338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:26.086470 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:26.087457 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:26.125496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:26.125925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:26.126913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:26.126913 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:26.126913 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:26.126913 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:26.126913 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:26.126913 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:26.126913 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:26.126913 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:26.128122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:26.128122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:26.128122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:26.128122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:26.128122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:26.128122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:26.128122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:26.128122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:26.128122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:26.128122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:26.128122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:26.128122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:26.128730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:26.128730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:26.128730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:26.128730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:26.128730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:26.128730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:26.128730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:26.128730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:26.128730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:26.128730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:26.128730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:26.128730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:26.128730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:26.129141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:26.129141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:26.129550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:26.129989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:26.130693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:26.130693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:26.130693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:28.777461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:28.778460 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:28.778554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:28.779332 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:28.779557 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:28.779716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.780319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:28.780319 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.780319 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:28.780319 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:28.780319 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.780319 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:28.780319 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.780319 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:28.781080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.781080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:28.781080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:28.781080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:28.781080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:28.781080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:28.781080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:28.781080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.781080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:28.781080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:28.781080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:28.781080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.781733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:28.781733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:28.781733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:28.781733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:28.781733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:28.781733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:28.781733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:28.781733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:28.781733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.781733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:28.781733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:28.781733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.781733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:28.782384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:28.783181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.784760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:28.784760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:28.784760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.784760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:28.784760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.784760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:28.784760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:28.787380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:28.787487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:28.787612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:47:28.788050 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:28.788359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.788972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:28.788972 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.788972 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:28.788972 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:28.788972 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.788972 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:28.788972 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.788972 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:28.789703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:28.790171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:28.790171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:28.790171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.790171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:28.790171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:28.790171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.790171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:28.790171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.790171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:28.790656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.790656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:28.790656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:28.790656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:28.790656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:28.790656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:28.790656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:28.790656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:47:28.790656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:28.790656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:28.790656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.790656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:28.791014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:28.791812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:28.792306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:28.792658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.792658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:28.792658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:28.792658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.792658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:28.792658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.792658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:28.792658 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:28.792658 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:28.792658 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:28.792851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:28.793001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.793655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:28.793655 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.793655 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:28.793655 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:28.793655 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.793655 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:28.793655 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.793655 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:28.794285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:28.794715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:28.794715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:28.794715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.794715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:28.794715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:28.794715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.794715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:28.794715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.794715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:28.795365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.795365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:28.795365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:28.795365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:28.795365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:28.795365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:28.795365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:28.795365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:28.795365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:28.795365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.795365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:28.795864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:28.796435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.796435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:28.796435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.796435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:28.796839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:28.797380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:28.797876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.797876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:28.797876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.797876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:28.798289 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:28.798520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:28.798671 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:28.798900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:28.799404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:28.799774 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:28.800024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.800756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:28.800756 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.800756 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:28.800756 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:28.800756 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.800756 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:28.800756 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.800756 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:28.801696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.801696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:28.801696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:28.801696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:28.801696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:28.801696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:28.801696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:28.801696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.801696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:28.801696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:28.801696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:28.801696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.802248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:28.802248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:28.802248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:28.802248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:28.802248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:28.802248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:28.802248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:28.802248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:28.802248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.802248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:28.802248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:28.802248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.802248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:28.802898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.802898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:28.803335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:28.803796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:28.804336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:28.804336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:28.804336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:31.086256 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:31.086977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:31.086977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:31.087619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:31.119608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:31.120011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:31.120982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:31.120982 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:31.120982 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:31.120982 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:31.120982 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:31.120982 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:31.120982 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:31.120982 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:31.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:31.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:31.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:31.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:31.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:31.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:31.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:31.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:31.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:31.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:31.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:31.121744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:31.122428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:31.122428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:31.122428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:31.122428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:31.122428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:31.122428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:31.122428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:31.122428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:31.122428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:31.122428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:31.122428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:31.122428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:31.122428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:31.123569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:31.124372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:31.125173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:31.125173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:31.125173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:31.125173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:31.125173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:31.125173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:33.779577 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:33.780561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:33.780561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:33.781050 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:33.781380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:33.781700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.782512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:33.782512 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.782512 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:33.782512 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:33.782512 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.782512 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:33.782512 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.782512 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:33.783674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.783674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:33.783674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:33.783674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:33.783674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:33.783674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:33.783674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:33.783674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.783674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:33.783674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:33.783674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:33.783674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.784150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:33.784150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:33.784150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:33.784150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:33.784150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:33.784150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:33.784150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:33.784150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:33.784150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.784150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:33.784150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:33.784150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.784150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:33.784659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.784659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:33.785196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:33.785691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.786193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:33.786193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.786193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:33.787514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:33.787835 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:33.787996 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:33.788167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:47:33.788615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:33.788899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.789666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:33.789666 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.789666 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:33.789666 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:33.789666 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.789666 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:33.789666 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.789666 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:33.790762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:33.791328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:33.791328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:33.791328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.791328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:33.791328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:33.791328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.791328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:33.791328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.791328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:33.791869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.791869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:33.791869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:33.791869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:33.791869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:33.791869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:33.791869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:33.791869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:47:33.791869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:33.791869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:33.791869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.791869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:33.792389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:33.793906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:33.794259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:33.794617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:33.794617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:33.794617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:33.794617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:33.794617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:33.794617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.794617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:33.794617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:33.794617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.794617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:33.794617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.794617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:33.795613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:33.796062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:33.796278 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:33.796552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:33.797556 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:33.797981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.798902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:33.798902 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.798902 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:33.798902 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:33.798902 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.798902 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:33.798902 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.798902 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:33.799369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:33.800513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:33.800795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:33.800795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:33.800795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.800795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:33.800795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:33.800795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.800795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:33.800795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.800795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:33.801455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.801455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:33.801455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:33.801455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:33.801455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:33.801455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:33.801455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:33.801455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:33.801455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:33.801455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.801455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:33.801700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:33.801968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.801968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:33.801968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.801968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:33.803064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:33.803538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:33.803927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.803927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:33.803927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.803927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:33.803927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:33.803927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:33.803927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:33.804143 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:33.804389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:33.804539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.805028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:33.805028 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.805028 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:33.805028 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:33.805028 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.805028 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:33.805028 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.805028 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:33.805947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.805947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:33.805947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:33.805947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:33.805947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:33.805947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:33.805947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:33.805947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.805947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:33.805947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:33.805947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:33.805947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.806201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:33.806201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:33.806201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:33.806201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:33.806201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:33.806201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:33.806201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:33.806201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:33.806201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.806201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:33.806201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:33.806201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.806201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:33.806464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.806464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:33.807018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:33.807345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:33.807856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:33.807856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:33.807856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:36.086749 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:36.087456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:36.087640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:36.088869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:36.121506 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:36.121824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:36.122743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:36.122743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:36.122743 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:36.122743 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:36.122743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:36.122743 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:36.122743 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:36.122743 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:36.123599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:36.123599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:36.123599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:36.123599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:36.123599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:36.123599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:36.123599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:36.123599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:36.123599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:36.123599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:36.123599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:36.123599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:36.124197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:36.124197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:36.124197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:36.124197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:36.124197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:36.124197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:36.124197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:36.124197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:36.124197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:36.124197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:36.124197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:36.124197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:36.124197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:36.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:36.125706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:36.126623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:36.126623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:36.126623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:36.126623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:36.126623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:36.126623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:38.780109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:38.781038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:38.781038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:38.782002 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:38.782304 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:38.782517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.783771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:38.783771 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.783771 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:38.783771 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:38.783771 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.783771 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:38.783771 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.783771 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:38.785386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.785386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:38.785386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:38.785386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:38.785386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:38.785386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:38.785386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:38.785386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.785386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:38.785386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:38.785386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:38.785386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.786091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:38.786091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:38.786091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:38.786091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:38.786091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:38.786091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:38.786091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:38.786091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:38.786091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.786091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:38.786091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:38.786091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.786091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:38.786658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.786658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:38.787606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:38.788331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.789254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:38.789254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.789254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:38.789254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:38.789254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:38.789254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:38.789254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:47:38.789254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:38.789502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.790103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:38.790103 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.790103 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:38.790103 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:38.790103 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.790103 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:38.790103 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.790103 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:38.790630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:38.790979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:38.790979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:38.790979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.790979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:38.790979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:38.790979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.790979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:38.790979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.790979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:38.791195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.791195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:38.791195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:38.791195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:38.791195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:38.791195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:38.791195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:38.791195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:47:38.791195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:38.791195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:38.791195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.791195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:38.791445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:38.791683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.791683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:38.791683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.791683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:38.791909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:38.792188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:38.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:38.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:38.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:38.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.792716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:38.796021 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:38.796340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:38.796419 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:38.796520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:38.797090 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:38.797519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:38.798344 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.798344 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:38.798344 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:38.798344 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.798344 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:38.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.798344 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:38.799120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:38.799402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:38.799402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:38.799402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.799402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:38.799402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:38.799402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.799402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:38.799402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.799402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:38.800069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.800069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:38.800069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:38.800069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:38.800069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:38.800069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:38.800069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:38.800069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:38.800069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:38.800069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.800069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:38.800300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:38.800532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.800532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:38.800532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.800532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:38.801253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:38.801557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:38.801880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.801880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:38.801880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.801880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:38.803355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:38.803813 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:38.803927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:38.804113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:38.804617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:38.805013 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:38.805252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.806014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:38.806014 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.806014 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:38.806014 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:38.806014 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.806014 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:38.806014 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.806014 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:38.806938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.806938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:38.806938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:38.806938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:38.806938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:38.806938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:38.806938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:38.806938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.806938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:38.806938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:38.806938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:38.806938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.807250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:38.807250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:38.807250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:38.807250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:38.807250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:38.807250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:38.807250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:38.807250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:38.807250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.807250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:38.807250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:38.807250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.807250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:38.807481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.807481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:38.807998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:38.808299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:38.808649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:38.808649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:38.808649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:41.087407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:41.087863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:41.087910 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:41.088657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:41.119326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:41.119548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:41.120379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:41.120379 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:41.120379 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:41.120379 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:41.120379 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:41.120379 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:41.120379 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:41.120379 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:41.121282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:41.121282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:41.121282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:41.121282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:41.121282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:41.121282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:41.121282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:41.121282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:41.121282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:41.121282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:41.121282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:41.121282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:41.121931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:41.121931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:41.121931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:41.121931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:41.121931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:41.121931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:41.121931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:41.121931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:41.121931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:41.121931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:41.121931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:41.121931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:41.121931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:41.122602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:41.123415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:41.124302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:41.124302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:41.124302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:41.124302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:41.124302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:41.124302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:43.780752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:43.781271 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:43.781324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:43.782287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:43.782643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:43.782875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.783714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:43.783714 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.783714 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:43.783714 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:43.783714 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.783714 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:43.783714 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.783714 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:43.784731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.784731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:43.784731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:43.784731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:43.784731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:43.784731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:43.784731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:43.784731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.784731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:43.784731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:43.784731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:43.784731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.785108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:43.785108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:43.785108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:43.785108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:43.785108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:43.785108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:43.785108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:43.785108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:43.785108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.785108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:43.785108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:43.785108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.785108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:43.785495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.785495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:43.785816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:43.786283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.786803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:43.786803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.786803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:43.788276 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:43.788545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:43.788654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:43.788767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:47:43.789278 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:43.789558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.790242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:43.790242 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.790242 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:43.790242 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:43.790242 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.790242 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:43.790242 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.790242 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:43.791592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:43.791592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:43.791592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.791592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:43.791592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:43.791592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.791592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:43.791592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.791592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:43.791901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.791901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:43.791901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:43.791901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:43.791901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:43.791901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:43.791901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:43.791901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:47:43.791901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:43.791901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:43.791901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.791901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:43.792290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:43.792700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.792700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:43.792700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.792700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:43.792825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:43.793303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:43.793861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.793861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:43.793861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:43.793861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.793861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:43.793861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.793861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:43.796362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:43.796694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:43.796801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:43.796920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:43.797427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:43.797770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.798547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:43.798547 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.798547 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:43.798547 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:43.798547 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.798547 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:43.798547 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.798547 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:43.799643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:43.800104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:43.800104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:43.800104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.800104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:43.800104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:43.800104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.800104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:43.800104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.800104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:43.800553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.800553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:43.800553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:43.800553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:43.800553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:43.800553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:43.800553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:43.800553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:43.800553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:43.800553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.800553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:43.800934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:43.801344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.801344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:43.801344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.801344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:43.801909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:43.802459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:43.803002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.803002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:43.803002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.803002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:43.803757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:43.803985 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:43.804089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:43.804272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:43.804716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:43.805061 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:43.805318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.806000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:43.806000 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.806000 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:43.806000 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:43.806000 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.806000 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:43.806000 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.806000 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:43.806898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.806898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:43.806898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:43.806898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:43.806898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:43.806898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:43.806898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:43.806898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.806898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:43.806898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:43.806898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:43.806898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.807330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:43.807330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:43.807330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:43.807330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:43.807330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:43.807330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:43.807330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:43.807330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:43.807330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.807330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:43.807330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:43.807330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.807330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:43.807728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.807728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:43.808107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:43.808611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:43.809170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:43.809170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:43.809170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:46.087880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:46.088851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:46.089042 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:46.090311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:46.126618 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:46.127024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:46.128005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:46.128005 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:46.128005 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:46.128005 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:46.128005 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:46.128005 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:46.128005 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:46.128005 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:46.128722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:46.128722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:46.128722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:46.128722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:46.128722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:46.128722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:46.128722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:46.128722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:46.128722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:46.128722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:46.128722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:46.128722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:46.129083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:46.129083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:46.129083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:46.129083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:46.129083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:46.129083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:46.129083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:46.129083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:46.129083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:46.129083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:46.129083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:46.129083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:46.129083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:46.129538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:46.130016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:46.130549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:46.130549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:46.130549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:46.130549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:46.130549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:46.130549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:48.781728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:48.782327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:48.782327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:48.783358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:48.783739 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:48.784024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.785016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:48.785016 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.785016 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:48.785016 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:48.785016 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.785016 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:48.785016 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.785016 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:48.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:48.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:48.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:48.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:48.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:48.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:48.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:48.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:48.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:48.787081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.787523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:48.787523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:48.787523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:48.787523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:48.787523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:48.787523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:48.787523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:48.787523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:48.787523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.787523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:48.787523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:48.787523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.787523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:48.787931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.787931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:48.789175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:48.789696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.790273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:48.790273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.790273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:48.790273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:48.790412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:48.790484 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:48.790676 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:47:48.791282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:48.791991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.792632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:48.792632 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.792632 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:48.792632 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:48.792632 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.792632 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:48.792632 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.792632 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:48.793151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:48.793645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:48.793645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:48.793645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.793645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:48.793645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:48.793645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.793645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:48.793645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.793645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:48.793987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.793987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:48.793987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:48.793987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:48.793987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:48.793987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:48.793987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:48.793987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:47:48.793987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:48.793987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:48.793987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.793987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:48.794391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:48.794797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.794797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:48.794797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.794797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:48.795330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:48.795876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:48.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:48.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:48.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:48.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.796432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:48.796745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:48.797082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:48.797082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:48.797177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:48.797566 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:48.797803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:48.798342 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.798342 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:48.798342 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:48.798342 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.798342 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:48.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:48.799052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:48.799665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:48.799665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:48.799665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.799665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:48.799665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:48.799665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.799665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:48.799665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.799665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:48.799936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.799936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:48.799936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:48.799936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:48.799936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:48.799936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:48.799936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:48.799936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:48.799936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:48.799936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.799936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:48.800302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:48.800677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.800677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:48.800677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.800677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:48.801003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:48.801515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:48.802026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.802026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:48.802026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.802026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:48.804045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:48.804632 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:48.804632 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:48.804632 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:48.805191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:48.805623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:48.806005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.806719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:48.806719 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.806719 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:48.806719 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:48.806719 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.806719 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:48.806719 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.806719 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:48.807540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.807540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:48.807540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:48.807540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:48.807540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:48.807540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:48.807540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:48.807540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.807540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:48.807540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:48.807540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:48.807540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.808128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:48.808128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:48.808128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:48.808128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:48.808128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:48.808128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:48.808128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:48.808128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:48.808128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.808128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:48.808128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:48.808128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.808128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:48.808536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.808536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:48.808608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:48.809049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:48.809556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:48.809556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:48.809556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:51.088912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:51.089663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:51.089710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:51.091073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:51.122736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:51.122971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:51.123583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:51.123583 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:51.123583 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:51.123583 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:51.123583 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:51.123583 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:51.123583 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:51.123583 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:51.124349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:51.124349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:51.124349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:51.124349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:51.124349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:51.124349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:51.124349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:51.124349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:51.124349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:51.124349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:51.124349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:51.124349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:51.124766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:51.124766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:51.124766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:51.124766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:51.124766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:51.124766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:51.124766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:51.124766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:51.124766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:51.124766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:51.124766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:51.124766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:51.124766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:51.125395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:51.125882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:51.126466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:51.126466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:51.126466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:51.126466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:51.126466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:51.126466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:53.782257 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:53.783047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:53.783047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:53.784003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:53.784402 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:53.784720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.785835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:53.785835 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.785835 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:53.785835 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:53.785835 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.785835 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:53.785835 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.785835 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:53.787373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.787373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:53.787373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:53.787373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:53.787373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:53.787373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:53.787373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:53.787373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.787373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:53.787373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:53.787373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:53.787373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.787925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:53.787925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:53.787925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:53.787925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:53.787925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:53.787925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:53.787925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:53.787925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:53.787925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.787925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:53.787925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:53.787925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.787925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:53.788504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.788504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:53.789032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:53.789745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.790319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:53.790319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.790319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:53.790407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:53.790739 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:53.790905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:53.791083 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:47:53.791476 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:53.791672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.792236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:53.792236 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.792236 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:53.792236 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:53.792236 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.792236 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:53.792236 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.792236 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:53.793179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:53.793715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:53.793715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:53.793715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.793715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:53.793715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:53.793715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.793715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:53.793715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.793715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:53.794064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.794064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:53.794064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:53.794064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:53.794064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:53.794064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:53.794064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:53.794064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:47:53.794064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:53.794064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:53.794064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.794064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:53.794518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:53.794892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.794892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:53.794892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.794892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:53.795103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:53.795593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:53.796102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.796102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:53.796102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:53.796102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.796102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:53.796102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.796102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:53.796988 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:53.797217 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:53.797331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:53.797457 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:53.797788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:53.798100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.798661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:53.798661 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.798661 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:53.798661 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:53.798661 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.798661 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:53.798661 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.798661 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:53.799406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:53.799820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:53.799820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:53.799820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.799820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:53.799820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:53.799820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.799820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:53.799820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.799820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:53.800312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.800312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:53.800312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:53.800312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:53.800312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:53.800312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:53.800312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:53.800312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:53.800312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:53.800312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.800312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:53.800693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:53.801057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.801057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:53.801057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.801057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:53.801567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:53.802105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:53.802671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.802671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:53.802671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.802671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:53.804337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:53.804576 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:53.804719 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:53.804877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:53.805804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:53.806085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:53.806274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.806804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:53.806804 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.806804 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:53.806804 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:53.806804 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.806804 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:53.806804 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.806804 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:53.807690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.807690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:53.807690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:53.807690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:53.807690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:53.807690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:53.807690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:53.807690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.807690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:53.807690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:53.807690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:53.807690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.808074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:53.808074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:53.808074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:53.808074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:53.808074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:53.808074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:53.808074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:53.808074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:53.808074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.808074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:53.808074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:53.808074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.808074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:53.808464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.808464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:53.808782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:53.809337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:53.809880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:53.809880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:53.809880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:56.089678 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:56.090496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:56.090496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:56.092023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:56.124935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:56.125319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:56.126073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:56.126073 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:56.126073 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:56.126073 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:56.126073 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:56.126073 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:56.126073 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:56.126073 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:56.126860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:56.126860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:56.126860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:56.126860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:56.126860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:56.126860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:56.126860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:56.126860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:56.126860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:56.126860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:56.126860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:56.126860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:56.127464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:56.127464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:56.127464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:56.127464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:56.127464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:56.127464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:56.127464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:56.127464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:56.127464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:56.127464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:56.127464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:56.127464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:56.127464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:56.128093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:56.128823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:56.129643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:56.129643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:56.129643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:56.129643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:56.129643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:56.129643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:58.782777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:58.783575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:58.783575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:58.784184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:58.784528 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:58.784757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.785489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:58.785489 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.785489 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:58.785489 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:58.785489 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.785489 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:58.785489 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.785489 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:58.786262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.786262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:58.786262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:58.786262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:58.786262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:58.786262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:58.786262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:58.786262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.786262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:58.786262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:58.786262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:58.786262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.786822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:58.786822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:58.786822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:58.786822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:58.786822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:58.786822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:58.786822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:58.786822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:58.786822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.786822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:58.786822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:58.786822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.786822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:58.787391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:58.788090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.788863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:58.788863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:58.788863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.788863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:58.788863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.788863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:58.790933 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:58.791120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:58.791192 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:58.791291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:47:58.791810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:58.792145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.792831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:58.792831 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.792831 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:58.792831 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:58.792831 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.792831 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:58.792831 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.792831 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:58.793937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:58.794382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:58.794382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:58.794382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.794382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:58.794382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:58.794382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.794382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:58.794382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.794382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:58.795192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.795192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:58.795192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:58.795192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:58.795192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:58.795192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:58.795192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:58.795192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:47:58.795192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:58.795192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:58.795192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.795192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:58.795548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:58.795918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.795918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:58.795918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.795918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:58.796802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:58.797250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:58.797711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.797711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:58.797711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:58.797711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.797711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:58.797711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.797711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:58.797711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:58.797711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:58.797993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:58.797993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:58.798514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:58.798857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.799589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:58.799589 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.799589 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:58.799589 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:58.799589 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.799589 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:58.799589 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.799589 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:47:58.800789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:47:58.801247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:58.801247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:47:58.801247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.801247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:47:58.801247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:47:58.801247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.801247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:47:58.801247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.801247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:47:58.801626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.801626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:58.801626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:58.801626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:58.801626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:58.801626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:58.801626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:58.801626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:58.801626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:58.801626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.801626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:58.801942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:58.802322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.802322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:58.802322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.802322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:58.802643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:58.803108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:58.803611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.803611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:58.803611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.803611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:47:58.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:47:58.804951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:47:58.805058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:47:58.805199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:47:58.805697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:58.806036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:47:58.806289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.806951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:47:58.806951 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.806951 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:47:58.806951 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:47:58.806951 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.806951 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:47:58.806951 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.806951 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:47:58.807745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.807745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:47:58.807745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:58.807745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:47:58.807745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:47:58.807745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:58.807745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:58.807745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.807745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:58.807745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:58.807745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:47:58.807745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.808100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:47:58.808100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:58.808100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:47:58.808100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:58.808100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:47:58.808100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:47:58.808100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:58.808100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:47:58.808100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.808100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:47:58.808100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:47:58.808100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.808100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:47:58.808463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.808463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:47:58.808772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:47:58.809220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:47:58.809704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:47:58.809704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:47:58.809704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:01.090397 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:01.091151 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:01.091151 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:01.092409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:01.128802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:01.129152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:01.130021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:01.130021 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:01.130021 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:01.130021 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:01.130021 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:01.130021 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:01.130021 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:01.130021 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:01.131019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:01.131019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:01.131019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:01.131019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:01.131019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:01.131019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:01.131019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:01.131019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:01.131019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:01.131019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:01.131019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:01.131019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:01.131966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:01.131966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:01.131966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:01.131966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:01.131966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:01.131966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:01.131966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:01.131966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:01.131966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:01.131966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:01.131966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:01.131966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:01.131966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:01.132683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:01.134535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:01.136178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:01.136178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:01.136178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:01.136178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:01.136178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:01.136178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:03.784440 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:03.784440 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:03.785668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:03.785830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:03.786042 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:03.786298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.787150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:03.787150 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.787150 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:03.787150 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:03.787150 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.787150 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:03.787150 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.787150 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:03.788674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.788674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:03.788674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:03.788674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:03.788674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:03.788674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:03.788674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:03.788674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.788674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:03.788674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:03.788674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:03.788674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.789348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:03.789348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:03.789348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:03.789348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:03.789348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:03.789348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:03.789348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:03.789348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:03.789348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.789348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:03.789348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:03.789348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.789348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:03.790066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.790066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:03.790401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:03.791498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.792640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:03.792640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.792640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:03.792640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:03.792640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:03.792640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:03.792640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:48:03.792995 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:03.793847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.794978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:03.794978 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.794978 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:03.794978 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:03.794978 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.794978 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:03.794978 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.794978 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:03.796004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:03.797471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:03.798704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:03.798704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:03.798704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:48:03.798704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:03.798704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:03.798704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.798704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:03.798704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:03.798704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:03.798704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.798704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:03.798704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:03.798704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:03.799699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:03.799699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:03.799699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:03.799699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:03.799699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:03.799699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.799699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:03.799699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:03.799699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.799699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:03.799699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.799699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:03.800081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:03.800745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:03.801671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.801671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:03.801671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:03.801671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.801671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:03.801671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.801671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:03.801671 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:03.801671 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:03.801671 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:03.802165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:03.802165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:03.802165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.802165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:03.802165 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.802165 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:03.802165 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:03.802165 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.802165 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:03.802165 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.802165 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:03.802165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:03.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.802948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:03.802948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:03.802948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.802948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:03.802948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.802948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:03.802948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.802948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:03.802948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:03.802948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:03.802948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:03.802948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:03.802948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:03.803341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:03.803341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:03.803341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.803341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:03.803341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:03.803341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:03.803341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.803341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:03.803341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:03.803341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:03.803341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:03.803341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:03.803341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:03.803709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:03.803709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:03.803709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.803709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:03.803709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:03.803709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.803709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:03.803709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.803709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:03.803928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:03.804289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:03.804624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.804624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:03.804624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.804624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:03.805060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:03.805303 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:03.805407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:03.805554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:03.805931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:03.806278 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:03.806484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.807091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:03.807091 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.807091 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:03.807091 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:03.807091 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.807091 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:03.807091 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.807091 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:03.807793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.807793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:03.807793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:03.807793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:03.807793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:03.807793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:03.807793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:03.807793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.807793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:03.807793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:03.807793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:03.807793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.808180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:03.808180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:03.808180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:03.808180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:03.808180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:03.808180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:03.808180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:03.808180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:03.808180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.808180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:03.808180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:03.808180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.808180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:03.808575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.808575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:03.808806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:03.809163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:03.809557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:03.809557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:03.809557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:06.091004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:06.091884 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:06.092068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:06.093239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:06.094884 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:48:06.120083 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:06.120480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:06.121319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:06.121319 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:06.121319 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:06.121319 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:06.121319 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:06.121319 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:06.121319 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:06.121319 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:06.122608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:06.122608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:06.122608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:06.122608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:06.122608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:06.122608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:06.122608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:06.122608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:06.122608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:06.122608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:06.122608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:06.122608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:06.123389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:06.123389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:06.123389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:06.123389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:06.123389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:06.123389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:06.123389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:06.123389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:06.123389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:06.123389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:06.123389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:06.123389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:06.123389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:06.124426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:06.125087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:06.125924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:06.125924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:06.125924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:06.125924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:06.125924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:06.125924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:08.783971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:08.784882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:08.784882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:08.785074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:08.785431 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:08.785678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.786626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:08.786626 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.786626 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:08.786626 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:08.786626 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.786626 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:08.786626 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.786626 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:08.787615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.787615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:08.787615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:08.787615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:08.787615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:08.787615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:08.787615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:08.787615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.787615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:08.787615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:08.787615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:08.787615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.788177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:08.788177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:08.788177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:08.788177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:08.788177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:08.788177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:08.788177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:08.788177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:08.788177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.788177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:08.788177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:08.788177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.788177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:08.788772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:08.789512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.790384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:08.790384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:08.790384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.790384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:08.790384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.790384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:08.791711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:08.791906 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:08.792006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:08.792135 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:48:08.792617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:08.793065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.793874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:08.793874 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.793874 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:08.793874 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:08.793874 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.793874 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:08.793874 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.793874 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:08.794618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:08.795063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:08.795063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:08.795063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.795063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:08.795063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:08.795063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.795063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:08.795063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.795063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:08.795410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.795410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:08.795410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:08.795410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:08.795410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:08.795410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:08.795410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:08.795410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:48:08.795410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:08.795410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:08.795410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.795410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:08.795778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:08.796196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.796196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:08.796196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.796196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:08.796396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:08.796870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:08.797319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.797319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:08.797319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:08.797319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.797319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:08.797319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.797319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:08.799515 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:08.799696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:08.799795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:08.799924 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:08.800417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:08.800843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.801484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:08.801484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.801484 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:08.801484 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:08.801484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.801484 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:08.801484 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.801484 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:08.802236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:08.802668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:08.802668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:08.802668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.802668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:08.802668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:08.802668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.802668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:08.802668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.802668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:08.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:08.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:08.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:08.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:08.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:08.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:08.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:08.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:08.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:08.803465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:08.803924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.803924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:08.803924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.803924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:08.804162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:08.804704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:08.805222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.805222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:08.805222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.805222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:08.805639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:08.805930 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:08.806028 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:08.806168 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:08.806550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:08.806875 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:08.807083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.807746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:08.807746 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.807746 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:08.807746 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:08.807746 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.807746 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:08.807746 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.807746 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:08.808579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.808579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:08.808579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:08.808579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:08.808579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:08.808579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:08.808579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:08.808579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.808579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:08.808579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:08.808579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:08.808579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.808947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:08.808947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:08.808947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:08.808947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:08.808947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:08.808947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:08.808947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:08.808947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:08.808947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.808947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:08.808947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:08.808947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.808947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:08.809334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.809334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:08.809515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:08.810233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:08.810807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:08.810807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:08.810807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:11.091700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:11.092252 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:11.092441 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:11.093597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:11.128675 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:11.129052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:11.129863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:11.129863 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:11.129863 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:11.129863 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:11.129863 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:11.129863 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:11.129863 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:11.129863 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:11.130645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:11.130645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:11.130645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:11.130645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:11.130645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:11.130645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:11.130645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:11.130645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:11.130645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:11.130645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:11.130645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:11.130645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:11.131179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:11.131179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:11.131179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:11.131179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:11.131179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:11.131179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:11.131179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:11.131179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:11.131179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:11.131179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:11.131179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:11.131179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:11.131179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:11.131711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:11.133306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:11.134112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:11.134112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:11.134112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:11.134112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:11.134112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:11.134112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:13.784418 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:13.785394 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:13.785394 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:13.786134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:13.786482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:13.786714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.787666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:13.787666 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.787666 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:13.787666 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:13.787666 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.787666 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:13.787666 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.787666 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:13.789173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.789173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:13.789173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:13.789173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:13.789173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:13.789173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:13.789173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:13.789173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.789173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:13.789173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:13.789173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:13.789173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.789873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:13.789873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:13.789873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:13.789873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:13.789873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:13.789873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:13.789873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:13.789873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:13.789873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.789873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:13.789873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:13.789873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.789873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:13.790480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.790480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:13.792947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:13.793663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.794321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:13.794321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.794321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:13.794321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:13.794321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:13.794321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:13.794321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:48:13.794563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:13.794865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.795605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:13.795605 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.795605 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:13.795605 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:13.795605 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.795605 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:13.795605 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.795605 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:13.796363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:13.796905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:13.796905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:13.796905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.796905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:13.796905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:13.796905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.796905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:13.796905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.796905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:13.797238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.797238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:13.797238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:13.797238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:13.797238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:13.797238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:13.797238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:13.797238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:48:13.797238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:13.797238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:13.797238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.797238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:13.797648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:13.798097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.798097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:13.798097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.798097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:13.798557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:13.799057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:13.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:13.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:13.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:13.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:13.799823 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:13.799917 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:13.800026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:13.800240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:13.800664 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:13.801038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.801784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:13.801784 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.801784 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:13.801784 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:13.801784 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.801784 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:13.801784 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.801784 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:13.802533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:13.802807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:13.802807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:13.802807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.802807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:13.802807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:13.802807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.802807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:13.802807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.802807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:13.803432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.803432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:13.803432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:13.803432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:13.803432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:13.803432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:13.803432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:13.803432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:13.803432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:13.803432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.803432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:13.803673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:13.803929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.803929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:13.803929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.803929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:13.804516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:13.804886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:13.805259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.805259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:13.805259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.805259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:13.805944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:13.806126 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:13.806257 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:13.806409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:13.806809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:13.807160 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:13.807432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.808086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:13.808086 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.808086 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:13.808086 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:13.808086 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.808086 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:13.808086 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.808086 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:13.808921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.808921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:13.808921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:13.808921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:13.808921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:13.808921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:13.808921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:13.808921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.808921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:13.808921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:13.808921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:13.808921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.809229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:13.809229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:13.809229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:13.809229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:13.809229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:13.809229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:13.809229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:13.809229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:13.809229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.809229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:13.809229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:13.809229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.809229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:13.809471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.809471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:13.810123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:13.810488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:13.810956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:13.810956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:13.810956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:16.096877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:16.097650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:16.097650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:16.099094 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:16.133540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:16.134157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:16.134897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:16.134897 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:16.134897 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:16.134897 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:16.134897 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:16.134897 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:16.134897 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:16.134897 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:16.136088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:16.136088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:16.136088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:16.136088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:16.136088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:16.136088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:16.136088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:16.136088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:16.136088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:16.136088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:16.136088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:16.136088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:16.136690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:16.136690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:16.136690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:16.136690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:16.136690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:16.136690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:16.136690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:16.136690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:16.136690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:16.136690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:16.136690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:16.136690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:16.136690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:16.137278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:16.137278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:16.137386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:16.138075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:16.138940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:16.138940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:16.138940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:18.784896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:18.785739 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:18.785739 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:18.786432 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:18.786710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:18.786942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.787685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:18.787685 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.787685 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:18.787685 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:18.787685 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.787685 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:18.787685 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.787685 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:18.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:18.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:18.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:18.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:18.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:18.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:18.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:18.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:18.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:18.788661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.789266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:18.789266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:18.789266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:18.789266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:18.789266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:18.789266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:18.789266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:18.789266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:18.789266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.789266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:18.789266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:18.789266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.789266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:18.789774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:18.790464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:18.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:18.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:18.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.791306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:18.793727 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:18.793959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:18.794075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:18.794198 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:48:18.794628 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:18.794901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.795551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:18.795551 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.795551 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:18.795551 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:18.795551 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.795551 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:18.795551 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.795551 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:18.796329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:18.796971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:18.796971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:18.796971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.796971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:18.796971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:18.796971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.796971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:18.796971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.796971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:18.797390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.797390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:18.797390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:18.797390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:18.797390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:18.797390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:18.797390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:18.797390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:48:18.797390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:18.797390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:18.797390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.797390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:18.797747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:18.798129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:18.798560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:18.799076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:18.799076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:18.799076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:18.799076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:18.799076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:18.799076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.799076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:18.799076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:18.799076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.799076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:18.799076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.799076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:18.800063 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:18.800501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:18.800686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:18.800879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:18.801359 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:18.801692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.802448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:18.802448 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.802448 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:18.802448 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:18.802448 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.802448 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:18.802448 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.802448 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:18.803610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:18.804301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:18.804301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:18.804301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.804301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:18.804301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:18.804301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.804301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:18.804301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.804301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:18.804997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.804997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:18.804997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:18.804997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:18.804997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:18.804997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:18.804997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:18.804997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:18.804997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:18.804997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.804997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:18.805361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:18.805763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.805763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:18.805763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.805763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:18.806993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:18.807468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:18.807969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.807969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:18.807969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.807969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:18.807969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:18.807969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:18.807969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:18.807969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:18.808232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:18.808232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:18.808376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.808855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:18.808855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.808855 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:18.808855 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:18.808855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.808855 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:18.808855 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.808855 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:18.809716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.809716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:18.809716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:18.809716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:18.809716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:18.809716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:18.809716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:18.809716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.809716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:18.809716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:18.809716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:18.809716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.810120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:18.810120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:18.810120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:18.810120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:18.810120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:18.810120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:18.810120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:18.810120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:18.810120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.810120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:18.810120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:18.810120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.810120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:18.810505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.810505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:18.810664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:18.811139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:18.811691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:18.811691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:18.811691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:21.097656 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:21.098343 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:21.098407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:21.100074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:21.163877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:21.163877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:21.164350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:21.164350 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:21.164350 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:21.164350 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:21.164350 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:21.164350 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:21.164350 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:21.164350 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:21.165083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.066 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:21.165083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:21.165083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:21.165083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:21.165083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:21.165083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:21.165083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:21.165083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:21.165083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:21.165083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:21.165083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:21.165083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:21.165373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:21.165373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:21.165373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:21.165373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:21.165373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:21.165373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:21.165373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:21.165373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:21.165373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:21.165373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:21.165373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:21.165373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:21.165373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:21.165634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:21.165634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:21.167275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:21.168077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:21.169059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:21.169059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:21.169059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:23.785573 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:23.786155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:23.786155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:23.787115 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:23.787420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:23.787647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.788446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:23.788446 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.788446 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:23.788446 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:23.788446 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.788446 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:23.788446 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.788446 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:23.789450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.789450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:23.789450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:23.789450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:23.789450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:23.789450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:23.789450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:23.789450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.789450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:23.789450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:23.789450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:23.789450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.789811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:23.789811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:23.789811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:23.789811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:23.789811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:23.789811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:23.789811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:23.789811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:23.789811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.789811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:23.789811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:23.789811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.789811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:23.790123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.790123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:23.791231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:23.791624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.792062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:23.792062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.792062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:23.794113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:23.794550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:23.794670 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:23.795237 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:48:23.795298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:23.795538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.796250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:23.796250 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.796250 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:23.796250 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:23.796250 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.796250 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:23.796250 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.796250 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:23.798447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:23.798793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:23.799096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:23.799096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:23.799096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:48:23.799096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:23.799096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:23.799096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.799096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:23.799096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:23.799096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:23.799096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.799096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:23.799096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:23.799096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:23.799366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:23.799366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:23.799366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:23.799366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:23.799366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:23.799366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.799366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:23.799366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:23.799366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.799366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:23.799366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.799366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:23.799613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:23.799903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:23.800261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.800261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:23.800261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:23.800261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.800261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:23.800261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.800261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:23.801143 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:23.801143 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:23.801143 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:23.801275 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:23.801707 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:23.802045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.802788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:23.802788 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.802788 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:23.802788 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:23.802788 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.802788 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:23.802788 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.802788 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:23.804054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:23.804374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:23.804374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:23.804374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.804374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:23.804374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:23.804374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.804374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:23.804374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.804374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:23.806392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.806392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:23.806392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:23.806392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:23.806392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:23.806392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:23.806392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:23.806392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:23.806392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:23.806392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.806392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:23.807078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:23.807720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:23.808344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:23.809139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:23.809139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:23.809139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:23.809139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:23.809139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.809139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:23.809139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:23.809139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.809139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:23.809139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.809139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:23.809139 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:23.809139 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:23.809778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:23.809778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:23.809778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:23.809778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:23.809778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.809778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:23.809778 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.809778 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:23.809778 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:23.810255 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.810255 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:23.810255 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.810255 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:23.810449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.810449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:23.810449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:23.810449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:23.810449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:23.810449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:23.810449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:23.810449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.810449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:23.810449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:23.810449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:23.810449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.810870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:23.810870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:23.810870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:23.810870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:23.810870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:23.810870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:23.810870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:23.810870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:23.810870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.810870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:23.810870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:23.810870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.810870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:23.811266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.811266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:23.812194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:23.812686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:23.813375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:23.813375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:23.813375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:26.098225 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:26.098918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:26.098918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:26.099401 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:26.129662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:26.129940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:26.130577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:26.130577 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:26.130577 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:26.130577 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:26.130577 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:26.130577 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:26.130577 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:26.130577 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:26.131248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:26.131248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:26.131248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:26.131248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:26.131248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:26.131248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:26.131248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:26.131248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:26.131248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:26.131248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:26.131248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:26.131248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:26.131608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:26.131608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:26.131608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:26.131608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:26.131608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:26.131608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:26.131608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:26.131608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:26.131608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:26.131608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:26.131608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:26.131608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:26.131608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:26.131994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:26.131994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:26.132071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:26.132526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:26.133107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:26.133107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:26.133107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:28.786249 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:28.787071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:28.787071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:28.788348 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:28.788683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:28.788935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.789875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:28.789875 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.789875 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:28.789875 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:28.789875 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.789875 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:28.789875 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.789875 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:28.791406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.792120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:28.792120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:28.792120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:28.792120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:28.792120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:28.792120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:28.792120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:28.792120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:28.792120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.792120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:28.792120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:28.792120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.792120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:28.793025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.793025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:28.794123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:28.795142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.796152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:28.796152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.796152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:28.796152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:28.796152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:28.796152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:28.796152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:48:28.796152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:28.796641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.796641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:28.796641 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.796641 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:28.796641 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:28.796641 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.796641 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:28.796641 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.796641 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:28.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:28.798143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:28.798143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:28.798143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.798143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:28.798143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:28.798143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.798143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:28.798143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.798143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:28.798748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.798748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:28.798748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:28.798748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:28.798748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:28.798748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:28.798748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:28.798748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:48:28.798748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:28.798748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:28.798748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.798748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:28.799388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:28.800359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.800359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:28.800359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.800359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:28.801421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:28.801931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:28.802298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.802298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:28.802298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:28.802298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.802298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:28.802298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.802298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:28.802298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:28.802298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:28.802298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:28.802510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:28.802510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:28.802568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.803095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:28.803095 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.803095 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:28.803095 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:28.803095 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.803095 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:28.803095 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.803095 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:28.804380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:28.804675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:28.804675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:28.804675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.804675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:28.804675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:28.804675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.804675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:28.804675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.804675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:28.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:28.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:28.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:28.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:28.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:28.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:28.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:28.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:28.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:28.806035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:28.806311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.806311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:28.806311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.806311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:28.807479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:28.807850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:28.808188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.808188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:28.808188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.808188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:28.808443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:28.808689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:28.808826 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:28.809045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:28.809503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:28.809838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:28.810058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.810896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:28.810896 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.810896 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:28.810896 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:28.810896 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.810896 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:28.810896 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.810896 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:28.812241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.812241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:28.812241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:28.812241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:28.812241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:28.812241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:28.812241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:28.812241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.812241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:28.812241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:28.812241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:28.812241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.812649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:28.812649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:28.812649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:28.812649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:28.812649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:28.812649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:28.812649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:28.812649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:28.812649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.812649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:28.812649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:28.812649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.812649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:28.813010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.813010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:28.813802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:28.814302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:28.814887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:28.814887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:28.814887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:31.098872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:31.099551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:31.099699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:31.100843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:31.134896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:31.135352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:31.136057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:31.136057 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:31.136057 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:31.136057 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:31.136057 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:31.136057 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:31.136057 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:31.136057 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:31.136720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:31.136720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:31.136720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:31.136720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:31.136720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:31.136720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:31.136720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:31.136720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:31.136720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:31.136720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:31.136720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:31.136720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:31.137101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:31.137101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:31.137101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:31.137101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:31.137101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:31.137101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:31.137101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:31.137101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:31.137101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:31.137101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:31.137101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:31.137101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:31.137101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:31.137467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:31.137893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:31.138401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:31.138401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:31.138401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:31.138401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:31.138401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:31.138401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:33.786796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:33.787576 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:33.787576 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:33.788458 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:33.788832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:33.789150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.789914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:33.789914 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.789914 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:33.789914 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:33.789914 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.789914 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:33.789914 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.789914 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:33.791007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.791007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:33.791007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:33.791007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:33.791007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:33.791007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:33.791007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:33.791007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.791007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:33.791007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:33.791007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:33.791007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.791587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:33.791587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:33.791587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:33.791587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:33.791587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:33.791587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:33.791587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:33.791587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:33.791587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.791587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:33.791587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:33.791587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.791587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:33.792051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.792051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:33.794200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:33.795105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.796118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:33.796118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.796118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:33.796118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:33.796118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:33.796118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:33.796118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:48:33.796118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:33.796364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.796618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:33.796618 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.796618 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:33.796618 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:33.796618 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.796618 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:33.796618 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.796618 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:33.797377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:33.797786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:33.797786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:33.797786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.797786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:33.797786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:33.797786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.797786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:33.797786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.797786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:33.798319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.798319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:33.798319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:33.798319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:33.798319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:33.798319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:33.798319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:33.798319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:48:33.798319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:33.798319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:33.798319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.798319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:33.799267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:33.800546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:33.800907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:33.801428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:33.801428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:33.801428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:33.801428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:33.801428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:33.801428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.801428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:33.801428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:33.801428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.801428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:33.801428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.801428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:33.802243 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:33.802639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:33.802639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:33.802639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:33.802927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:33.803120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.803575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:33.803575 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.803575 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:33.803575 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:33.803575 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.803575 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:33.803575 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.803575 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:33.804319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:33.804882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:33.804882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:33.804882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.804882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:33.804882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:33.804882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.804882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:33.804882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.804882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:33.805397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.805397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:33.805397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:33.805397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:33.805397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:33.805397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:33.805397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:33.805397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:33.805397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:33.805397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.805397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:33.805849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:33.806303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.806303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:33.806303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.806303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:33.806555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:33.806860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:33.807177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.807177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:33.807177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.807177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:33.809008 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:33.809377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:33.809536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:33.809725 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:33.810016 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:33.810224 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:33.810377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.810854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:33.810854 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.810854 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:33.810854 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:33.810854 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.810854 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:33.810854 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.810854 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:33.811815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.811815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:33.811815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:33.811815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:33.811815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:33.811815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:33.811815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:33.811815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.811815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:33.811815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:33.811815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:33.811815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.812252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:33.812252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:33.812252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:33.812252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:33.812252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:33.812252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:33.812252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:33.812252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:33.812252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.812252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:33.812252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:33.812252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.812252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:33.812554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.812554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:33.812941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:33.813381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:33.813844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:33.813844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:33.813844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:36.099625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:36.100360 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:36.100606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:36.101917 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:36.137800 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:36.138218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:36.139129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:36.139129 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:36.139129 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:36.139129 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:36.139129 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:36.139129 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:36.139129 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:36.139129 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:36.140330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:36.140330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:36.140330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:36.140330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:36.140330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:36.140330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:36.140330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:36.140330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:36.140330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:36.140330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:36.140330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:36.140330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:36.140869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:36.140869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:36.140869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:36.140869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:36.140869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:36.140869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:36.140869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:36.140869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:36.140869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:36.140869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:36.140869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:36.140869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:36.140869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:36.141439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:36.141439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:36.141867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:36.142368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:36.142940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:36.142940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:36.142940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:38.787662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:38.788567 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:38.788567 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:38.789435 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:38.789826 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:38.790085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.790952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:38.790952 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.790952 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:38.790952 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:38.790952 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.790952 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:38.790952 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.790952 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:38.791965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.791965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:38.791965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:38.791965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:38.791965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:38.791965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:38.791965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:38.791965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.791965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:38.791965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:38.791965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:38.791965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.792966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:38.792966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:38.792966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:38.792966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:38.792966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:38.792966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:38.792966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:38.792966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:38.792966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.792966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:38.792966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:38.792966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.792966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:38.793657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:38.794747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.795706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:38.795706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:38.795706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.795706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:38.795706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.795706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:38.795706 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:38.796095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:38.796095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:38.796260 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:48:38.796600 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:38.796808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.797328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:38.797328 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.797328 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:38.797328 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:38.797328 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.797328 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:38.797328 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.797328 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:38.797910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:38.798485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:38.798485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:38.798485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.798485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:38.798485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:38.798485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.798485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:38.798485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.798485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:38.798795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.798795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:38.798795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:38.798795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:38.798795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:38.798795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:38.798795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:38.798795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:48:38.798795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:38.798795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:38.798795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.798795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:38.799153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:38.799573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.799573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:38.799573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.799573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:38.799762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:38.800245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:38.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:38.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:38.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:38.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:38.802606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:38.802959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:38.803109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:38.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:38.803755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:38.804107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.804868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:38.804868 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.804868 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:38.804868 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:38.804868 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.804868 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:38.804868 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.804868 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:38.805652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:38.806102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:38.806102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:38.806102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.806102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:38.806102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:38.806102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.806102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:38.806102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.806102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:38.806581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.806581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:38.806581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:38.806581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:38.806581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:38.806581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:38.806581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:38.806581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:38.806581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:38.806581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.806581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:38.806937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:38.807352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.807352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:38.807352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.807352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:38.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:38.808181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:38.808738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.808738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:38.808738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.808738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:38.809495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:38.809799 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:38.809918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:38.810081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:38.810576 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:38.810930 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:38.811141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.811944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:38.811944 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.811944 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:38.811944 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:38.811944 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.811944 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:38.811944 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.811944 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:38.812912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.812912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:38.812912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:38.812912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:38.812912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:38.812912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:38.812912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:38.812912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.812912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:38.812912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:38.812912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:38.812912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.813332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:38.813332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:38.813332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:38.813332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:38.813332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:38.813332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:38.813332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:38.813332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:38.813332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.813332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:38.813332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:38.813332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.813332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:38.813699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.813699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:38.813903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:38.814399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:38.814928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:38.814928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:38.814928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:41.100486 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:41.102674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:41.102674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:41.102674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:41.138307 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:41.138307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:41.139408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:41.139408 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:41.139408 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:41.139408 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:41.139408 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:41.139408 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:41.139408 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:41.139408 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:41.140475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:41.140475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:41.140475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:41.140475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:41.140475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:41.140475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:41.140475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:41.140475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:41.140475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:41.140475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:41.140475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:41.140475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:41.141090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:41.141090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:41.141090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:41.141090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:41.141090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:41.141090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:41.141090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:41.141090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:41.141090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:41.141090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:41.141090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:41.141090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:41.141090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:41.141964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:41.142790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:41.143637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:41.143637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:41.143637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:41.143637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:41.143637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:41.143637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:43.788100 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:43.788703 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:43.788836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:43.790163 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:43.790702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:43.791034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.792132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:43.792132 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.792132 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:43.792132 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:43.792132 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.792132 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:43.792132 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.792132 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:43.793296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.793296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:43.793296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:43.793296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:43.793296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:43.793296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:43.793296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:43.793296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.793296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:43.793296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:43.793296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:43.793296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.793617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:43.793617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:43.793617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:43.793617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:43.793617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:43.793617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:43.793617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:43.793617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:43.793617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.793617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:43.793617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:43.793617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.793617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:43.793925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.793925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:43.794521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:43.797120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.797962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:43.797962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.797962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:43.797962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:43.797962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:43.797962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:43.797962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:48:43.797962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:43.798327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.799032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:43.799032 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.799032 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:43.799032 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:43.799032 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.799032 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:43.799032 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.799032 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:43.799918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:43.800595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:43.800595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:43.800595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.800595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:43.800595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:43.800595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.800595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:43.800595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.800595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:43.800837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.800837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:43.800837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:43.800837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:43.800837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:43.800837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:43.800837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:43.800837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:48:43.800837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:43.800837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:43.800837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.800837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:43.801199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:43.801590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.801590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:43.801590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.801590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:43.801760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:43.802258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:43.802739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.802739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:43.802739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:43.802739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.802739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:43.802739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.802739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:43.803146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:43.803462 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:43.803560 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:43.803705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:43.804145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:43.804612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.805330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:43.805330 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.805330 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:43.805330 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:43.805330 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.805330 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:43.805330 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.805330 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:43.806228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:43.806766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:43.806766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:43.806766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.806766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:43.806766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:43.806766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.806766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:43.806766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.806766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:43.807104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.807104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:43.807104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:43.807104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:43.807104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:43.807104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:43.807104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:43.807104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:43.807104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:43.807104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.807104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:43.807515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:43.807931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.807931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:43.807931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.807931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:43.808190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:43.808740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:43.809308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.809308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:43.809308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.809308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:43.810009 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:43.810384 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:43.810521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:43.810679 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:43.811169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:43.811605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:43.812447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.813406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:43.813406 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.813406 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:43.813406 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:43.813406 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.813406 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:43.813406 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.813406 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:43.814283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.814283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:43.814283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:43.814283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:43.814283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:43.814283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:43.814283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:43.814283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.814283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:43.814283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:43.814283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:43.814283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.814663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:43.814663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:43.814663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:43.814663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:43.814663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:43.814663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:43.814663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:43.814663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:43.814663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.814663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:43.814663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:43.814663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.814663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:43.815057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.815057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:43.815463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:43.815935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:43.816471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:43.816471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:43.816471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:46.101285 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:46.102753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:46.102919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:46.104431 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:46.144365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:46.144687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:46.145632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:46.145632 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:46.145632 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:46.145632 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:46.145632 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:46.145632 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:46.145632 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:46.145632 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:46.146885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:46.146885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:46.146885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:46.146885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:46.146885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:46.146885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:46.146885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:46.146885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:46.146885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:46.146885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:46.146885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:46.146885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:46.147295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:46.147295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:46.147295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:46.147295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:46.147295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:46.147295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:46.147295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:46.147295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:46.147295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:46.147295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:46.147295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:46.147295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:46.147295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:46.147658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:46.147658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:46.147956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:46.148772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:46.149410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:46.149410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:46.149410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:48.792190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:48.792190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:48.793029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:48.793645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:48.793961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:48.794090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.794894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:48.794894 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.794894 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:48.794894 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:48.794894 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.794894 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:48.794894 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.794894 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:48.796513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.796513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:48.796513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:48.796513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:48.796513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:48.796513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:48.796513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:48.796513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.796513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:48.796513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:48.796513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:48.796513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.797147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:48.797147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:48.797147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:48.797147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:48.797147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:48.797147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:48.797147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:48.797147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:48.797147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.797147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:48.797147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:48.797147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.797147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:48.797814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.797814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:48.798519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:48.799825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:48.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:48.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:48.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:48.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:48.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:48:48.800764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:48.801182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.801182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:48.801182 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.801182 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:48.801182 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:48.801182 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.801182 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:48.801182 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.801182 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:48.802419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:48.802872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:48.802872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:48.802872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.802872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:48.802872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:48.802872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.802872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:48.802872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.802872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:48.803725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.803725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:48.803725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:48.803725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:48.803725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:48.803725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:48.803725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:48.803725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:48:48.803725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:48.803725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:48.803725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.803725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:48.804084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:48.804435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.804435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:48.804435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.804435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:48.805442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:48.805913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:48.806411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.806411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:48.806411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:48.806411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.806411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:48.806411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.806411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:48.806411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:48.806411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:48.806411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:48.806939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:48.807394 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:48.807651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.808223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:48.808223 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.808223 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:48.808223 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:48.808223 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.808223 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:48.808223 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.808223 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:48.808964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:48.809357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:48.809357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:48.809357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.809357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:48.809357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:48.809357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.809357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:48.809357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.809357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:48.809823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.809823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:48.809823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:48.809823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:48.809823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:48.809823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:48.809823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:48.809823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:48.809823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:48.809823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.809823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:48.810170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:48.810549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.810549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:48.810549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.810549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:48.811000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:48.811501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:48.811983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.811983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:48.811983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.811983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:48.811983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:48.811983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:48.811983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:48.811983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:48.812238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:48.812238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:48.812462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.812953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:48.812953 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.812953 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:48.812953 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:48.812953 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.812953 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:48.812953 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.812953 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:48.813579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.813579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:48.813579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:48.813579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:48.813579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:48.813579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:48.813579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:48.813579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.813579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:48.813579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:48.813579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:48.813579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.813947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:48.813947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:48.813947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:48.813947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:48.813947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:48.813947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:48.813947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:48.813947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:48.813947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.813947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:48.813947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:48.813947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.813947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:48.814290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:48.814697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:48.815228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:48.815228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:48.815228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:48.815228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:48.815228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:48.815228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:51.102132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:51.102761 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:51.103008 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:51.104435 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:51.141820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:51.142469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:51.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:51.143789 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:51.143789 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:51.143789 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:51.143789 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:51.143789 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:51.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:51.143789 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:51.144973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:51.144973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:51.144973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:51.144973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:51.144973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:51.144973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:51.144973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:51.144973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:51.144973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:51.144973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:51.144973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:51.144973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:51.145323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:51.145323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:51.145323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:51.145323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:51.145323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:51.145323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:51.145323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:51.145323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:51.145323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:51.145323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:51.145323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:51.145323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:51.145323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:51.145643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:51.145643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:51.146130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:51.146556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:51.147022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:51.147022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:51.147022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:53.792370 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:53.793713 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:53.793713 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:53.794010 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:53.794272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:53.794395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.795025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:53.795025 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.795025 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:53.795025 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:53.795025 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.795025 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:53.795025 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.795025 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:53.796672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.796672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:53.796672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:53.796672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:53.796672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:53.796672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:53.796672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:53.796672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.796672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:53.796672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:53.796672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:53.796672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.797424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:53.797424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:53.797424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:53.797424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:53.797424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:53.797424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:53.797424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:53.797424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:53.797424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.797424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:53.797424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:53.797424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.797424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:53.797991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.797991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:53.798536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:53.799619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.800922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:53.800922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.800922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:53.800922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:53.800922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:53.800922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:53.800922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:48:53.800922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:53.801248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.801248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:53.801248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.801248 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:53.801248 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:53.801248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.801248 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:53.801248 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.801248 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:53.801638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:53.802303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:53.802303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:53.802303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.802303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:53.802303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:53.802303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.802303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:53.802303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.802303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:53.802852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.802852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:53.802852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:53.802852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:53.802852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:53.802852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:53.802852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:53.802852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:48:53.802852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:53.802852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:53.802852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.802852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:53.803646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:53.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:53.804581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:53.805036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:53.805036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:53.805036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:53.805036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:53.805036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:53.805036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.805036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:53.805036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:53.805036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.805036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:53.805036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.805036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:53.806027 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:53.806317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:53.806422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:53.806550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:53.806911 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:53.807149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.807691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:53.807691 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.807691 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:53.807691 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:53.807691 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.807691 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:53.807691 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.807691 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:53.808411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:53.809226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:53.809226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:53.809226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.809226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:53.809226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:53.809226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.809226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:53.809226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.809226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:53.809722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.809722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:53.809722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:53.809722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:53.809722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:53.809722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:53.809722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:53.809722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:53.809722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:53.809722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.809722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:53.810374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:53.811114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:53.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:53.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:53.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:53.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:53.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:53.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:53.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:53.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:53.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:53.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:53.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:53.812340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:53.812340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:53.812340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:53.812589 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:53.812734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:53.813248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.813248 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:53.813248 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:53.813248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.813248 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:53.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:53.814086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.814086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:53.814086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:53.814086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:53.814086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:53.814086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:53.814086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:53.814086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.814086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:53.814086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:53.814086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:53.814086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.814518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:53.814518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:53.814518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:53.814518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:53.814518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:53.814518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:53.814518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:53.814518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:53.814518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.814518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:53.814518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:53.814518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.814518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:53.814948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.814948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:53.815083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:53.815841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:53.816392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:53.816392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:53.816392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:56.102945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:56.103906 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:56.103906 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:56.104773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:56.140469 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:56.140856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:56.141709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:56.141709 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:56.141709 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:56.141709 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:56.141709 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:56.141709 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:56.141709 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:56.141709 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:56.142749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:56.142749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:56.142749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:56.142749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:56.142749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:56.142749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:56.142749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:56.142749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:56.142749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:56.142749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:56.142749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:56.142749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:56.143375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:56.143375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:56.143375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:56.143375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:56.143375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:56.143375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:56.143375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:56.143375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:56.143375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:56.143375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:56.143375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:56.143375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:56.143375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:56.143917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:56.144749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:56.145588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:56.145588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:56.145588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:56.145588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:56.145588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:56.145588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:58.792986 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:58.794070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:58.794070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:58.794477 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:58.794836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:58.795091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.795925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:58.795925 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.795925 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:58.795925 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:58.795925 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.795925 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:58.795925 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.795925 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:58.796708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.796708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:58.796708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:58.796708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:58.796708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:58.796708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:58.796708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:58.796708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.796708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:58.796708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:58.796708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:58.796708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.797413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:58.797413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:58.797413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:58.797413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:58.797413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:58.797413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:58.797413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:58.797413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:58.797413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.797413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:58.797413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:58.797413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.797413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:58.798071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:58.798939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.799943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:58.799943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:58.799943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.799943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:58.799943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.799943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:58.799943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:58.799943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:58.799943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:58.799943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:48:58.800531 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:58.800718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.801452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:58.801452 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.801452 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:58.801452 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:58.801452 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.801452 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:58.801452 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.801452 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:58.802517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:58.803030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:58.803537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:58.803537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:58.803537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:48:58.803537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:58.803537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:58.803537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.803537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:58.803537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:58.803537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:58.803537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.803537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:58.803537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:58.803537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:58.803964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:58.803964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:58.803964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:58.803964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:58.803964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:58.803964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.803964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:58.803964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:58.803964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.803964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:58.803964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.803964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:58.804348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:58.804927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:58.805617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.805617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:58.805617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:58.805617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.805617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:58.805617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.805617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:58.807614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:58.807840 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:58.807932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:58.808029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:58.808588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:58.809006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.810954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:58.810954 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.810954 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:58.810954 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:58.810954 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.810954 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:58.810954 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.810954 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:48:58.811562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:48:58.811949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:58.811949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:48:58.811949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.811949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:48:58.811949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:48:58.811949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.811949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:48:58.811949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.811949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:48:58.813619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.813619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:58.813619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:58.813619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:58.813619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:58.813619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:58.813619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:58.813619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:58.813619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:58.813619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.813619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:58.813933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:58.814304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.814304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:58.814304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.814304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:58.814304 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:58.815503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:58.816061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:58.816597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.816597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:58.816597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.816597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:48:58.816597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:48:58.816597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:48:58.816597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:48:58.816597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:58.816873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:48:58.817108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.819338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:48:58.819338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.819338 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:48:58.819338 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:48:58.819338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.819338 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:48:58.819338 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.819338 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:48:58.819600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.819600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:48:58.819600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:58.819600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:48:58.819600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:48:58.819600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:58.819600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:58.819600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.819600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:58.819600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:58.819600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:48:58.819600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.819977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:48:58.819977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:58.819977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:48:58.819977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:58.819977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:48:58.819977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:48:58.819977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:58.819977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:48:58.819977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.819977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:48:58.819977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:48:58.819977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.819977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:48:58.820305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.820305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:48:58.820615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:48:58.821055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:48:58.821737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:48:58.821737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:48:58.821737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:01.103677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:01.104339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:01.104520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:01.105749 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:01.145080 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:01.145463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:01.146292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:01.146292 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:01.146292 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:01.146292 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:01.146292 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:01.146292 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:01.146292 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:01.146292 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:01.147039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:01.147039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:01.147039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:01.147039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:01.147039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:01.147039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:01.147039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:01.147039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:01.147039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:01.147039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:01.147039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:01.147039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:01.147593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:01.147593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:01.147593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:01.147593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:01.147593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:01.147593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:01.147593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:01.147593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:01.147593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:01.147593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:01.147593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:01.147593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:01.147593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:01.147886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:01.148414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:01.148858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:01.148858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:01.148858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:01.148858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:01.148858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:01.148858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:03.793645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:03.794727 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:03.794727 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:03.795484 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:03.795683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:03.795920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.796764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:03.796764 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.796764 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:03.796764 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:03.796764 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.796764 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:03.796764 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.796764 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:03.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:03.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:03.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:03.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:03.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:03.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:03.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:03.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:03.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:03.798342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.798980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:03.798980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:03.798980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:03.798980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:03.798980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:03.798980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:03.798980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:03.798980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:03.798980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.798980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:03.798980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:03.798980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.798980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:03.799679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.799679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:03.800108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:03.800959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.802101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:03.802101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.802101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:03.802101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:03.802101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:03.802101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:03.802101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:49:03.802101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:03.802604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.802604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:03.802604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.802604 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:03.802604 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:03.802604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.802604 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:03.802604 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.802604 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:03.803138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:03.804147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:03.804862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:03.804862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:03.804862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:49:03.804862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:03.804862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:03.804862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.804862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:03.804862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:03.804862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:03.804862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.804862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:03.804862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:03.804862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:03.805645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:03.805645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:03.805645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:03.805645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:03.805645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:03.805645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.805645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:03.805645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:03.805645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.805645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:03.805645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.805645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:03.806051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:03.806765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:03.807262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.807262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:03.807262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:03.807262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.807262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:03.807262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.807262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:03.808769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:03.808769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:03.808769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:03.808769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:03.808974 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:03.809060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.809587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:03.809587 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.809587 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:03.809587 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:03.809587 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.809587 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:03.809587 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.809587 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:03.810247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:03.810669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:03.810669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:03.810669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.810669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:03.810669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:03.810669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.810669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:03.810669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.810669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:03.811105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.811105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:03.811105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:03.811105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:03.811105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:03.811105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:03.811105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:03.811105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:03.811105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:03.811105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.811105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:03.811462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:03.811849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.811849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:03.811849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.811849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:03.812160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:03.812678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:03.813193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.813193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:03.813193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.813193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:03.815810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:03.816076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:03.816177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:03.816344 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:03.816591 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:03.816790 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:03.816966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.817636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:03.817636 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.817636 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:03.817636 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:03.817636 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.817636 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:03.817636 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.817636 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:03.818411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.818411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:03.818411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:03.818411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:03.818411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:03.818411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:03.818411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:03.818411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.818411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:03.818411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:03.818411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:03.818411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.818778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:03.818778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:03.818778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:03.818778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:03.818778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:03.818778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:03.818778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:03.818778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:03.818778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.818778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:03.818778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:03.818778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.818778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:03.819138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.819138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:03.819349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:03.819810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:03.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:03.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:03.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:06.104512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:06.105863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:06.106441 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:06.107319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:06.109260 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:49:06.140531 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:06.140907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:06.142365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:06.142365 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:06.142365 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:06.142365 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:06.142365 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:06.142365 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:06.142365 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:06.142365 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:06.142853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:06.142853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:06.142853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:06.142853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:06.142853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:06.142853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:06.142853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:06.142853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:06.142853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:06.142853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:06.142853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:06.142853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:06.143575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:06.143575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:06.143575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:06.143575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:06.143575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:06.143575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:06.143575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:06.143575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:06.143575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:06.143575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:06.143575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:06.143575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:06.143575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:06.144151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:06.145088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:06.145909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:06.145909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:06.145909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:06.145909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:06.145909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:06.145909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:08.794463 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:08.795350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:08.795350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:08.795874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:08.796056 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:08.796194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.796690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:08.796690 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.796690 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:08.796690 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:08.796690 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.796690 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:08.796690 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.796690 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:08.797662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.797662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:08.797662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:08.797662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:08.797662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:08.797662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:08.797662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:08.797662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.797662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:08.797662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:08.797662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:08.797662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:08.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:08.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:08.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:08.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:08.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:08.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:08.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:08.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:08.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:08.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:08.798889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:08.799561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.800390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:08.800390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:08.800390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.800390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:08.800390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.800390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:08.800874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:08.801173 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:08.801317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:08.801456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:49:08.801706 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:08.801958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.802470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:08.802470 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.802470 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:08.802470 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:08.802470 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.802470 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:08.802470 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.802470 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:08.803269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:08.803650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:08.803650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:08.803650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.803650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:08.803650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:08.803650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.803650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:08.803650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.803650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:08.804135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.804135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:08.804135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:08.804135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:08.804135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:08.804135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:08.804135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:08.804135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:49:08.804135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:08.804135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:08.804135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.804135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:08.804465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:08.804816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.804816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:08.804816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.804816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:08.805218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:08.805608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:08.806067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.806067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:08.806067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:08.806067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.806067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:08.806067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.806067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:08.808321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:08.808568 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:08.808688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:08.808823 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:08.809246 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:08.809601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.810305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:08.810305 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.810305 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:08.810305 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:08.810305 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.810305 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:08.810305 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.810305 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:08.811121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:08.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:08.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:08.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:08.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:08.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:08.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.811558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:08.812053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.812053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:08.812053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:08.812053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:08.812053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:08.812053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:08.812053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:08.812053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:08.812053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:08.812053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.812053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:08.812380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:08.812741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.812741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:08.812741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.812741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:08.813168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:08.813629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:08.814112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.814112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:08.814112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.814112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:08.816135 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:08.816386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:08.816496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:08.816665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:08.817034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:08.817407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:08.817626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.818200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:08.818200 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.818200 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:08.818200 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:08.818200 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.818200 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:08.818200 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.818200 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:08.819086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.819086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:08.819086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:08.819086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:08.819086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:08.819086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:08.819086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:08.819086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.819086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:08.819086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:08.819086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:08.819086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.819446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:08.819446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:08.819446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:08.819446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:08.819446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:08.819446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:08.819446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:08.819446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:08.819446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.819446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:08.819446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:08.819446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.819446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:08.819797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.819797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:08.820100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:08.820521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:08.821012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:08.821012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:08.821012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:11.106041 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:11.106923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:11.106923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:11.107554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:11.140413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:11.140969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:11.142115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:11.142115 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:11.142115 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:11.142115 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:11.142115 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:11.142115 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:11.142115 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:11.142115 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:11.143094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:11.143094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:11.143094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:11.143094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:11.143094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:11.143094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:11.143094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:11.143094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:11.143094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:11.143094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:11.143094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:11.143094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:11.143750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:11.143750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:11.143750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:11.143750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:11.143750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:11.143750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:11.143750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:11.143750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:11.143750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:11.143750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:11.143750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:11.143750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:11.143750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:11.144583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:11.145585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:11.146633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:11.146633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:11.146633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:11.146633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:11.146633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:11.146633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:13.795087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:13.795978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:13.795978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:13.796623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:13.796821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:13.796987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.797807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:13.797807 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.797807 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:13.797807 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:13.797807 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.797807 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:13.797807 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.797807 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:13.799303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.799303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:13.799303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:13.799303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:13.799303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:13.799303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:13.799303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:13.799303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.799303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:13.799303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:13.799303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:13.799303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.800185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:13.800185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:13.800185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:13.800185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:13.800185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:13.800185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:13.800185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:13.800185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:13.800185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.800185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:13.800185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:13.800185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.800185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:13.800724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.800724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:13.801175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:13.801936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.802597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:13.802597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.802597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:13.802597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:13.802597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:13.802597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:13.802597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:49:13.802954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:13.803287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.803963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:13.803963 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.803963 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:13.803963 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:13.803963 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.803963 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:13.803963 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.803963 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:13.804843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:13.805552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:13.805552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:13.805552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.805552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:13.805552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:13.805552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.805552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:13.805552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.805552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:13.806066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.806066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:13.806066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:13.806066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:13.806066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:13.806066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:13.806066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:13.806066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:49:13.806066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:13.806066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:13.806066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.806066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:13.806796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:13.807522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.807522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:13.807522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.807522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:13.807839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:13.808288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:13.808727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.808727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:13.808727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:13.808727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.808727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:13.808727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.808727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:13.808727 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:13.808981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:13.809025 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:13.809133 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:13.809506 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:13.809821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.810363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:13.810363 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.810363 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:13.810363 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:13.810363 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.810363 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:13.810363 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.810363 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:13.811025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:13.811599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:13.811599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:13.811599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.811599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:13.811599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:13.811599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.811599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:13.811599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.811599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:13.812086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.812086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:13.812086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:13.812086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:13.812086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:13.812086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:13.812086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:13.812086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:13.812086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:13.812086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.812086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:13.812815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:13.813240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:13.813615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:13.813941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:13.813941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:13.813941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:13.813941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:13.813941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.813941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:13.813941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:13.813941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.813941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:13.813941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.813941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:13.816849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:13.817105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:13.817253 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:13.817498 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:13.817816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:13.818069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:13.818213 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.818714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:13.818714 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.818714 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:13.818714 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:13.818714 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.818714 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:13.818714 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.818714 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:13.820158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.820158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:13.820158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:13.820158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:13.820158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:13.820158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:13.820158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:13.820158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.820158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:13.820158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:13.820158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:13.820158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:13.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:13.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:13.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:13.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:13.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:13.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:13.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:13.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:13.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:13.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:13.821518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:13.822014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:13.822593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:13.822593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:13.822593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:16.106670 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:16.107308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:16.107567 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:16.108851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:16.143957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:16.144386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:16.145391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:16.145391 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:16.145391 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:16.145391 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:16.145391 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:16.145391 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:16.145391 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:16.145391 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:16.146074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:16.146074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:16.146074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:16.146074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:16.146074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:16.146074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:16.146074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:16.146074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:16.146074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:16.146074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:16.146074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:16.146074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:16.146650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:16.146650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:16.146650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:16.146650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:16.146650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:16.146650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:16.146650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:16.146650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:16.146650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:16.146650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:16.146650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:16.146650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:16.146650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:16.147194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:16.147863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:16.148667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:16.148667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:16.148667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:16.148667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:16.148667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:16.148667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:18.795744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:18.796622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:18.796622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:18.797425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:18.797775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:18.798025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.798735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:18.798735 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.798735 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:18.798735 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:18.798735 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.798735 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:18.798735 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.798735 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:18.800128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.800128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:18.800128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:18.800128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:18.800128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:18.800128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:18.800128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:18.800128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.800128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:18.800128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:18.800128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:18.800128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.800727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:18.800727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:18.800727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:18.800727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:18.800727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:18.800727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:18.800727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:18.800727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:18.800727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.800727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:18.800727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:18.800727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.800727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:18.801455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.801455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:18.801754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:18.802445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.802974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:18.802974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.802974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:18.802974 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:18.805283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:18.805283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:18.805283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:49:18.805766 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:18.805964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.806712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:18.806712 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.806712 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:18.806712 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:18.806712 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.806712 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:18.806712 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.806712 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:18.807353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:18.807821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:18.808262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:18.808262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:18.808262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:49:18.808262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:18.808262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:18.808262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.808262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:18.808262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:18.808262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:18.808262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.808262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:18.808262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:18.808262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:18.808656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:18.808656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:18.808656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:18.808656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:18.808656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:18.808656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.808656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:18.808656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:18.808656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.808656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:18.808656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.808656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:18.809000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:18.809480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:18.809967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.809967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:18.809967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:18.809967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.809967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:18.809967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.809967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:18.809967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:18.809967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:18.809967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:18.810320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:18.810320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:18.810320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.810692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:18.810692 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.810692 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:18.810692 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:18.810692 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.810692 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:18.810692 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.810692 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:18.811187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:18.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:18.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:18.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:18.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:18.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:18.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.811486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:18.811793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.811793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:18.811793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:18.811793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:18.811793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:18.811793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:18.811793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:18.811793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:18.811793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:18.811793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.811793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:18.812014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:18.812292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.812292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:18.812292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.812292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:18.812523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:18.812850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:18.813190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.813190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:18.813190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.813190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:18.817313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:18.818010 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:18.818148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:18.818321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:18.818658 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:18.818973 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:18.819174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.819778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:18.819778 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.819778 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:18.819778 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:18.819778 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.819778 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:18.819778 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.819778 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:18.821219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.821219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:18.821219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:18.821219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:18.821219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:18.821219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:18.821219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:18.821219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.821219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:18.821219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:18.821219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:18.821219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.821570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:18.821570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:18.821570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:18.821570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:18.821570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:18.821570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:18.821570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:18.821570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:18.821570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.821570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:18.821570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:18.821570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.821570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:18.821921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.821921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:18.822296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:18.822751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:18.823258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:18.823258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:18.823258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:21.107381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:21.108192 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:21.108192 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:21.109194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:21.136679 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:21.137178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:21.138046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:21.138046 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:21.138046 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:21.138046 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:21.138046 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:21.138046 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:21.138046 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:21.138046 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:21.138795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:21.138795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:21.138795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:21.138795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:21.138795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:21.138795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:21.138795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:21.138795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:21.138795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:21.138795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:21.138795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:21.138795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:21.139823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:21.139823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:21.139823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:21.139823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:21.139823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:21.139823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:21.139823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:21.139823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:21.139823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:21.139823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:21.139823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:21.139823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:21.139823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:21.140780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:21.141740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:21.142628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:21.142628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:21.142628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:21.142628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:21.142628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:21.142628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:23.796634 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:23.798886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:23.798886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:23.798886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:23.812289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:23.813467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:23.814499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:23.814499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:23.814499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.814499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:23.814499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:23.814499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.814499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:23.814499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.814499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:23.814499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.814499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:23.814499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:23.815485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:23.816810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:23.817426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:23.817426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:23.817426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:49:23.817426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:23.817426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.817426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:23.817426 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.817426 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:23.817426 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:23.817764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.818520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:23.819013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:23.819013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:23.819013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:23.819013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:23.819013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:23.819013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:49:23.819013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:23.819013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:23.819013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.819013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:23.819013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:23.819013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:23.819013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.819472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:23.819472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:23.819472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:23.819472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:23.819472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:23.819472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:23.819472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:23.819472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:23.819472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.819472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:23.819472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:23.819472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.819472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:23.819878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:23.820337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:23.820926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:23.820926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:23.820926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.820926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:23.820926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:23.820926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.820926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:23.820926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.820926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:23.820926 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:23.820926 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:23.820926 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:23.821355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:23.821355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:23.821355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.821355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:23.821355 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.821355 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:23.821355 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:23.821355 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.821355 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:23.821355 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.821355 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:23.821355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:23.821731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.822313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:23.822313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:23.822313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.822313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:23.822313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.822313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:23.822313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.822313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:23.822313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:23.822313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:23.822313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:23.822313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:23.822313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:23.822907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:23.822907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:23.822907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.822907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:23.822907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:23.822907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:23.822907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.822907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:23.822907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:23.822907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:23.822907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:23.822907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:23.822907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:23.823445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:23.823445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:23.823445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.823445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:23.823445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:23.823445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.823445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:23.823445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.823445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:23.823445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.823445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:23.823445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.824023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.824803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:23.825546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:23.825546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:23.825546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:23.825546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:23.825546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:23.825546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:23.825846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:23.826522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:23.827033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.827033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:23.827033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:23.827033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.827033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:23.827033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.827033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:23.827033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.827033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:23.827033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:23.827033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:23.827033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:23.827033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:23.827585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:23.828423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:23.828423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:23.828423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:23.828423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:23.828423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:23.828423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:23.828423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:23.828423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:23.828423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:23.828423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:23.828423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:23.828423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:26.108029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:26.108849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:26.108849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:26.110236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:26.149899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:26.151325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:26.151748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:26.151748 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:26.151748 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:26.151748 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:26.151748 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:26.151748 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:26.151748 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:26.151748 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:26.152785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:26.152785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:26.152785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:26.152785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:26.152785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:26.152785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:26.152785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:26.152785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:26.152785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:26.152785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:26.152785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:26.152785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:26.153086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:26.153086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:26.153086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:26.153086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:26.153086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:26.153086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:26.153086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:26.153086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:26.153086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:26.153086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:26.153086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:26.153086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:26.153086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:26.153373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:26.153373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:26.153796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:26.154099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:26.154464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:26.154464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:26.154464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:28.797154 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:28.797908 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:28.797908 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:28.798749 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:28.799072 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:28.799332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.800145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:28.800145 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.800145 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:28.800145 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:28.800145 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.800145 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:28.800145 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.800145 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:28.801746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.801746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:28.801746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:28.801746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:28.801746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:28.801746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:28.801746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:28.801746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.801746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:28.801746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:28.801746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:28.801746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.802032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:28.802032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:28.802032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:28.802032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:28.802032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:28.802032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:28.802032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:28.802032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:28.802032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.802032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:28.802032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:28.802032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.802032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:28.802299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.802299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:28.802750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:28.803123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.803503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:28.803503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.803503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:28.805318 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:28.805553 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:28.805662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:28.805776 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:49:28.806258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:28.806580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.807286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:28.807286 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.807286 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:28.807286 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:28.807286 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.807286 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:28.807286 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.807286 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:28.808141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:28.809100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:28.810678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:28.810678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:28.810678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:49:28.810678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:28.810678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:28.810678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.810678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:28.810678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:28.810678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:28.810678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.810678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:28.810678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:28.810678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:28.811280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:28.811280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:28.811280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:28.811280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:28.811280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:28.811280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.811280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:28.811280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:28.811280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.811280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:28.811280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.811280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:28.811945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:28.813625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:28.814417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.814417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:28.814417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:28.814417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.814417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:28.814417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.814417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:28.814417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:28.814417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:28.814417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:28.814729 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:28.814729 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:28.814976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.815499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:28.815499 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.815499 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:28.815499 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:28.815499 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.815499 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:28.815499 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.815499 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:28.816232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:28.816797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:28.816797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:28.816797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.816797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:28.816797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:28.816797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.816797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:28.816797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.816797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:28.817097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.817097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:28.817097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:28.817097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:28.817097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:28.817097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:28.817097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:28.817097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:28.817097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:28.817097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.817097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:28.817519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:28.818010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.818010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:28.818010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.818010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:28.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:28.818807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:28.819836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.819836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:28.819836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.819836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:28.819836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:28.819836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:28.819836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:28.819836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:28.820109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:28.820109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:28.820226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.820744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:28.820744 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.820744 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:28.820744 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:28.820744 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.820744 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:28.820744 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.820744 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:28.821556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.821556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:28.821556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:28.821556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:28.821556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:28.821556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:28.821556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:28.821556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.821556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:28.821556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:28.821556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:28.821556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.822069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:28.822069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:28.822069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:28.822069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:28.822069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:28.822069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:28.822069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:28.822069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:28.822069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.822069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:28.822069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:28.822069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.822069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:28.822572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:28.823084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:28.823640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:28.823640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:28.823640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:28.823640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:28.823640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:28.823640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:31.108629 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:31.109072 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:31.109201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:31.110045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:31.143491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:31.143753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:31.144378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:31.144378 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:31.144378 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:31.144378 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:31.144378 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:31.144378 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:31.144378 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:31.144378 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:31.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:31.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:31.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:31.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:31.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:31.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:31.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:31.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:31.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:31.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:31.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:31.145039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:31.145787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:31.145787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:31.145787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:31.145787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:31.145787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:31.145787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:31.145787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:31.145787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:31.145787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:31.145787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:31.145787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:31.145787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:31.145787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:31.146471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:31.147784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:31.148803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:31.148803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:31.148803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:31.148803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:31.148803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:31.148803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:33.797681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:33.798282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:33.798432 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:33.800037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:33.800640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:33.800993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.802140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:33.802140 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.802140 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:33.802140 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:33.802140 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.802140 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:33.802140 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.802140 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:33.803824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.803824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:33.803824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:33.803824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:33.803824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:33.803824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:33.803824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:33.803824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.803824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:33.803824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:33.803824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:33.803824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.804096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:33.804096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:33.804096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:33.804096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:33.804096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:33.804096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:33.804096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:33.804096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:33.804096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.804096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:33.804096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:33.804096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.804096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:33.804361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.804361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:33.804835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:33.805157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.805547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:33.805547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.805547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:33.805720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:33.805981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:33.806081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:33.806196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:49:33.806784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:33.807055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:33.807657 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.807657 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:33.807657 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:33.807657 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.807657 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:33.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.807657 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:33.808511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:33.808863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:33.808863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:33.808863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.808863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:33.808863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:33.808863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.808863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:33.808863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.808863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:33.809321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.809321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:33.809321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:33.809321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:33.809321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:33.809321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:33.809321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:33.809321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:49:33.809321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:33.809321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:33.809321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.809321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:33.809605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:33.809878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.809878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:33.809878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.809878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:33.810322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:33.810657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:33.811047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.811047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:33.811047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:33.811047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.811047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:33.811047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.811047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:33.814062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:33.814316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:33.814456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:33.814581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:33.815491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:33.815729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.816296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:33.816296 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.816296 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:33.816296 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:33.816296 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.816296 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:33.816296 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.816296 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:33.817024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:33.817354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:33.817354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:33.817354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.817354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:33.817354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:33.817354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.817354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:33.817354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.817354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:33.817911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.817911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:33.817911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:33.817911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:33.817911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:33.817911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:33.817911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:33.817911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:33.817911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:33.817911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.817911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:33.818172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:33.818481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.818481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:33.818481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.818481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:33.818993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:33.819383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:33.819771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.819771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:33.819771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.819771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:33.819771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:33.819771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:33.819771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:33.819771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:33.820057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:33.820270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:33.820431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.820966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:33.820966 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.820966 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:33.820966 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:33.820966 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.820966 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:33.820966 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.820966 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:33.821826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.821826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:33.821826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:33.821826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:33.821826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:33.821826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:33.821826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:33.821826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.821826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:33.821826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:33.821826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:33.821826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.822115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:33.822115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:33.822115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:33.822115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:33.822115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:33.822115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:33.822115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:33.822115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:33.822115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.822115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:33.822115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:33.822115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.822115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:33.822385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.822385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:33.822785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:33.823318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:33.823680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:33.823680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:33.823680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:36.109583 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:36.110439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:36.110439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:36.111553 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:36.151231 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:36.151231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:36.151494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:36.151494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:36.151494 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:36.151494 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:36.151494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:36.151494 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:36.151494 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:36.151494 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:36.152589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:36.152589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:36.152589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:36.152589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:36.152589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:36.152589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:36.152589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:36.152589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:36.152589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:36.152589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:36.152589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:36.152589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:36.153185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:36.153185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:36.153185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:36.153185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:36.153185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:36.153185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:36.153185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:36.153185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:36.153185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:36.153185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:36.153185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:36.153185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:36.153185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:36.153628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:36.153628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:36.154041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:36.154575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:36.155138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:36.155138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:36.155138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:38.798598 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:38.799479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:38.799479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:38.801655 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:38.801655 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:38.801655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.801885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:38.801885 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.801885 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:38.801885 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:38.801885 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.801885 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:38.801885 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.801885 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:38.803436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.803436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:38.803436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:38.803436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:38.803436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:38.803436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:38.803436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:38.803436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.803436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:38.803436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:38.803436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:38.803436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.804023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:38.804023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:38.804023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:38.804023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:38.804023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:38.804023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:38.804023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:38.804023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:38.804023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.804023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:38.804023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:38.804023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.804023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:38.804596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.804596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:38.805111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:38.805797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.806562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:38.806562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.806562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:38.806562 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:38.806747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:38.806929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:38.807108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:49:38.808346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:38.808346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.808626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:38.808626 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.808626 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:38.808626 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:38.808626 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.808626 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:38.808626 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.808626 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:38.809919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:38.810551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:38.810551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:38.810551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.810551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:38.810551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:38.810551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.810551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:38.810551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.810551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:38.811324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.811324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:38.811324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:38.811324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:38.811324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:38.811324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:38.811324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:38.811324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:49:38.811324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:38.811324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:38.811324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.811324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:38.811749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:38.812143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.812143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:38.812143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.812143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:38.813129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:38.813608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:38.814125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.814125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:38.814125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:38.814125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.814125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:38.814125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.814125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:38.814370 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:38.814690 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:38.814874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:38.815015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:38.815347 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:38.815694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.816291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:38.816291 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.816291 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:38.816291 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:38.816291 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.816291 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:38.816291 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.816291 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:38.817103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:38.817548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:38.817548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:38.817548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.817548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:38.817548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:38.817548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.817548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:38.817548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.817548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:38.817920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.817920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:38.817920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:38.817920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:38.817920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:38.817920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:38.817920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:38.817920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:38.817920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:38.817920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.817920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:38.818269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:38.818636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.818636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:38.818636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.818636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:38.818962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:38.819471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:38.819990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.819990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:38.819990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.819990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:38.819990 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:38.819990 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:38.819990 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:38.819990 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:38.820791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:38.821068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:38.821498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.822070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:38.822070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.822070 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:38.822070 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:38.822070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.822070 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:38.822070 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.822070 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:38.822870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.822870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:38.822870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:38.822870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:38.822870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:38.822870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:38.822870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:38.822870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.822870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:38.822870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:38.822870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:38.822870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.823240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:38.823240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:38.823240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:38.823240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:38.823240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:38.823240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:38.823240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:38.823240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:38.823240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.823240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:38.823240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:38.823240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.823240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:38.823591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.823591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:38.823821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:38.824328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:38.824884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:38.824884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:38.824884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:41.110758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:41.111573 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:41.111573 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:41.112816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:41.148741 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:41.149013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:41.149688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:41.149688 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:41.149688 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:41.149688 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:41.149688 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:41.149688 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:41.149688 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:41.149688 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:41.150433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:41.150433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:41.150433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:41.150433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:41.150433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:41.150433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:41.150433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:41.150433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:41.150433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:41.150433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:41.150433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:41.150433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:41.151137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:41.151137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:41.151137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:41.151137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:41.151137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:41.151137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:41.151137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:41.151137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:41.151137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:41.151137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:41.151137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:41.151137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:41.151137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:41.151899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:41.152746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:41.153773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:41.153773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:41.153773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:41.153773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:41.153773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:41.153773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:43.799386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:43.800274 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:43.800274 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:43.801371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:43.801696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:43.801960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.802849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:43.802849 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.802849 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:43.802849 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:43.802849 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.802849 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:43.802849 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.802849 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:43.804411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.804411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:43.804411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:43.804411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:43.804411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:43.804411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:43.804411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:43.804411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.804411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:43.804411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:43.804411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:43.804411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.804984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:43.804984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:43.804984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:43.804984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:43.804984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:43.804984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:43.804984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:43.804984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:43.804984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.804984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:43.804984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:43.804984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.804984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:43.805572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.805572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:43.806004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:43.806732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.807525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:43.807525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.807525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:43.807525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:43.807525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:43.807525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:43.807525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:49:43.807877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:43.808038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.808643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:43.808643 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.808643 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:43.808643 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:43.808643 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.808643 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:43.808643 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.808643 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:43.809353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:43.809838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:43.809838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:43.809838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.809838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:43.809838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:43.809838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.809838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:43.809838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.809838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:43.810121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.810121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:43.810121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:43.810121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:43.810121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:43.810121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:43.810121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:43.810121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:49:43.810121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:43.810121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:43.810121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.810121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:43.810495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:43.810874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.810874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:43.810874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.810874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:43.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:43.811546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:43.812029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.812029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:43.812029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:43.812029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.812029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:43.812029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.812029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:43.814880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:43.815102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:43.815216 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:43.815345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:43.815727 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:43.815989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.816539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:43.816539 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.816539 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:43.816539 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:43.816539 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.816539 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:43.816539 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.816539 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:43.817223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:43.817623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:43.817623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:43.817623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.817623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:43.817623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:43.817623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.817623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:43.817623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.817623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:43.818033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.818033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:43.818033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:43.818033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:43.818033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:43.818033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:43.818033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:43.818033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:43.818033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:43.818033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.818033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:43.818372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:43.818729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.818729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:43.818729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.818729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:43.819042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:43.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:43.820028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.820028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:43.820028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.820028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:43.820028 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:43.820028 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:43.820232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:43.820232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:43.820584 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:43.820825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:43.821020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.821510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:43.821510 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.821510 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:43.821510 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:43.821510 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.821510 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:43.821510 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.821510 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:43.822301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.822301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:43.822301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:43.822301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:43.822301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:43.822301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:43.822301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:43.822301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.822301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:43.822301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:43.822301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:43.822301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.822656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:43.822656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:43.822656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:43.822656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:43.822656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:43.822656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:43.822656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:43.822656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:43.822656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.822656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:43.822656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:43.822656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.822656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:43.822996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.822996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:43.823199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:43.823653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:43.824145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:43.824145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:43.824145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:46.111357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:46.112134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:46.112134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:46.113739 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:46.147056 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:46.147382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:46.147869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:46.147869 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:46.147869 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:46.147869 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:46.147869 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:46.147869 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:46.147869 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:46.147869 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:46.148767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:46.148767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:46.148767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:46.148767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:46.148767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:46.148767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:46.148767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:46.148767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:46.148767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:46.148767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:46.148767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:46.148767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:46.149165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:46.149165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:46.149165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:46.149165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:46.149165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:46.149165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:46.149165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:46.149165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:46.149165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:46.149165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:46.149165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:46.149165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:46.149165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:46.149590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:46.149590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:46.149916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:46.150463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:46.151114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:46.151114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:46.151114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:48.800460 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:48.801018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:48.801018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:48.801744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:48.801921 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:48.802057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.802641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:48.802641 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.802641 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:48.802641 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:48.802641 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.802641 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:48.802641 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.802641 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:48.803637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.803637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:48.803637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:48.803637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:48.803637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:48.803637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:48.803637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:48.803637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.803637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:48.803637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:48.803637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:48.803637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.803983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:48.803983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:48.803983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:48.803983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:48.803983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:48.803983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:48.803983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:48.803983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:48.803983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.803983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:48.803983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:48.803983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.803983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:48.804331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.804331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:48.804890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:48.805328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.805792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:48.805792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.805792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:48.806997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:48.807276 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:48.807410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:48.807532 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:49:48.807817 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:48.807976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.808425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:48.808425 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.808425 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:48.808425 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:48.808425 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.808425 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:48.808425 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.808425 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:48.809156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:48.809593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:48.809593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:48.809593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.809593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:48.809593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:48.809593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.809593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:48.809593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.809593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:48.810017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.810017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:48.810017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:48.810017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:48.810017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:48.810017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:48.810017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:48.810017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:49:48.810017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:48.810017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:48.810017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.810017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:48.810390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:48.810747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.810747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:48.810747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.810747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:48.811070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:48.811494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:48.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:48.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:48.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:48.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.811998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:48.815809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:48.816111 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:48.816242 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:48.816376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:48.816645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:48.816834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.817270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:48.817270 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.817270 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:48.817270 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:48.817270 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.817270 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:48.817270 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.817270 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:48.817983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:48.818407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:48.818407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:48.818407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.818407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:48.818407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:48.818407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.818407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:48.818407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.818407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:48.818899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.818899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:48.818899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:48.818899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:48.818899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:48.818899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:48.818899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:48.818899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:48.818899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:48.818899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.818899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:48.819254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:48.819625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.819625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:48.819625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.819625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:48.820020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:48.820492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:48.820953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.820953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:48.820953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.820953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:48.820953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:48.820953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:48.820953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:48.820953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:48.821190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:48.821346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:48.821466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.821872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:48.821872 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.821872 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:48.821872 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:48.821872 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.821872 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:48.821872 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.821872 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:48.822688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.822688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:48.822688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:48.822688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:48.822688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:48.822688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:48.822688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:48.822688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.822688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:48.822688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:48.822688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:48.822688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.823033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:48.823033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:48.823033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:48.823033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:48.823033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:48.823033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:48.823033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:48.823033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:48.823033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.823033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:48.823033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:48.823033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.823033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:48.823387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.823387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:48.823682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:48.824137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:48.824624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:48.824624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:48.824624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:51.111931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:51.112566 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:51.112690 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:51.113939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:51.147336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:51.147785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:51.148744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:51.148744 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:51.148744 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:51.148744 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:51.148744 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:51.148744 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:51.148744 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:51.148744 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:51.149749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:51.149749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:51.149749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:51.149749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:51.149749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:51.149749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:51.149749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:51.149749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:51.149749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:51.149749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:51.149749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:51.149749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:51.150327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:51.150327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:51.150327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:51.150327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:51.150327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:51.150327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:51.150327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:51.150327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:51.150327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:51.150327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:51.150327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:51.150327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:51.150327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:51.150873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:51.151634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:51.152453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:51.152453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:51.152453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:51.152453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:51.152453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:51.152453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:53.801168 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:53.802283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:53.802283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:53.803593 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:53.803914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:53.804155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.805258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:53.805258 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.805258 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:53.805258 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:53.805258 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.805258 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:53.805258 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.805258 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:53.806999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.806999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:53.806999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:53.806999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:53.806999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:53.806999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:53.806999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:53.806999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.806999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:53.806999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:53.806999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:53.806999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.807595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:53.807595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:53.807595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:53.807595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:53.807595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:53.807595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:53.807595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:53.807595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:53.807595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.807595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:53.807595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:53.807595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.807595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:53.808110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.808110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:53.808694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:53.809243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.809935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:53.809935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.809935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:53.809935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:53.809935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:53.809935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:53.809935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:49:53.810335 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:53.810596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.811463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:53.811463 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.811463 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:53.811463 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:53.811463 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.811463 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:53.811463 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.811463 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:53.814248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:53.815563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:53.816385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:53.816385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:53.816385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:49:53.816385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:53.816385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:53.816385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.816385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:53.816385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:53.816385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:53.816385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.816385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:53.816385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:53.816385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:53.817062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:53.817062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:53.817062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:53.817062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:53.817062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:53.817062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.817062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:53.817062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:53.817062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.817062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:53.817062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.817062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:53.817589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:53.817966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:53.818672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.818672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:53.818672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:53.818672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.818672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:53.818672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.818672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:53.818672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:53.818672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:53.818672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:53.819004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:53.819004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:53.819004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.819004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:53.819004 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.819004 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:53.819004 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:53.819004 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.819004 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:53.819004 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.819004 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:53.819004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:53.819407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:53.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:53.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:53.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:53.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:53.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:53.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:53.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:53.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:53.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:53.820727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:53.820727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:53.820727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.820727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:53.820727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:53.820727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:53.820727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.820727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:53.820727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:53.820727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:53.820727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:53.820727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:53.820727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:53.823418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:53.823418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:53.823418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.823418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:53.823418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:53.823418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.823418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:53.823418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.823418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:53.823418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.823418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:53.823418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.824517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.826148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:53.826927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:53.826927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:53.826927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:53.826927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:53.826927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:53.826927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:53.827534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:53.828018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:53.828512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.828512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:53.828512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:53.828512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.828512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:53.828512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.828512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:53.828512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.828512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:53.828512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:53.828512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:53.828512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:53.828512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:53.829014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:53.829742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:53.829742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:53.829742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:53.829742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:53.829742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:53.829742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:53.829742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:53.829742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:53.829742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:53.829742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:53.829742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:53.829742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:56.112757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:56.113718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:56.113718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:56.113920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:56.148447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:56.148778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:56.149626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:56.149626 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:56.149626 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:56.149626 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:56.149626 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:56.149626 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:56.149626 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:56.149626 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:56.150455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:56.150455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:56.150455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:56.150455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:56.150455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:56.150455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:56.150455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:56.150455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:56.150455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:56.150455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:56.150455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:56.150455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:56.151063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:56.151063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:56.151063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:56.151063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:56.151063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:56.151063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:56.151063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:56.151063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:56.151063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:56.151063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:56.151063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:56.151063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:56.151063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:56.151580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:56.152001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:56.152537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:56.152537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:56.152537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:56.152537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:56.152537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:56.152537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:58.802034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:58.802630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:58.802826 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:58.803542 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:58.803819 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:58.804072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:58.804827 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.804827 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:58.804827 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:58.804827 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.804827 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:58.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.804827 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:58.805722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.805722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:58.805722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:58.805722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:58.805722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:58.805722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:58.805722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:58.805722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.805722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:58.805722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:58.805722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:58.805722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.806068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:58.806068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:58.806068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:58.806068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:58.806068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:58.806068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:58.806068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:58.806068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:58.806068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.806068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:58.806068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:58.806068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.806068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:58.806409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.806409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:58.806785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:58.807228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.807744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:58.807744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.807744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:58.809273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:58.809551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:58.809661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:58.809791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:49:58.810174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:58.810478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.811089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:58.811089 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.811089 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:58.811089 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:58.811089 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.811089 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:58.811089 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.811089 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:58.811822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:58.812296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:58.812296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:58.812296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.812296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:58.812296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:58.812296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.812296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:58.812296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.812296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:58.812699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.812699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:58.812699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:58.812699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:58.812699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:58.812699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:58.812699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:58.812699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:49:58.812699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:58.812699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:58.812699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.812699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:58.813039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:58.813418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.813418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:58.813418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.813418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:58.813708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:58.814168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:58.814667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.814667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:58.814667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:58.814667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.814667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:58.814667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.814667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:58.816642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:58.816962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:58.817067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:58.817198 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:58.817595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:58.817937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.818579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:58.818579 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.818579 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:58.818579 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:58.818579 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.818579 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:58.818579 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.818579 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:49:58.819301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:49:58.819757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:58.819757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:49:58.819757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.819757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:49:58.819757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:49:58.819757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.819757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:49:58.819757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.819757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:49:58.820151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.820151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:58.820151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:58.820151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:58.820151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:58.820151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:58.820151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:58.820151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:58.820151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:58.820151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.820151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:58.820483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:58.820846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.820846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:58.820846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.820846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:58.821224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:58.821762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:58.822305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.822305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:58.822305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.822305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:49:58.822305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:49:58.822448 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:49:58.822552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:49:58.822686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:49:58.823037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:58.823393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:49:58.823617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.824391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:49:58.824391 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.824391 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:49:58.824391 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:49:58.824391 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.824391 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:49:58.824391 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.824391 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:49:58.825626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.825626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:49:58.825626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:58.825626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:49:58.825626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:49:58.825626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:58.825626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:58.825626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.825626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:58.825626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:58.825626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:49:58.825626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.826344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:49:58.826344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:58.826344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:49:58.826344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:58.826344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:49:58.826344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:49:58.826344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:58.826344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:49:58.826344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.826344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:49:58.826344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:49:58.826344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.826344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:49:58.826916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:49:58.827634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:49:58.828779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:49:58.828779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:49:58.828779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:49:58.828779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:49:58.828779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:49:58.828779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:01.113352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:01.114148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:01.114148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:01.115052 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:01.147492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:01.147955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:01.148851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:01.148851 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:01.148851 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:01.148851 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:01.148851 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:01.148851 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:01.148851 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:01.148851 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:01.149937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:01.149937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:01.149937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:01.149937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:01.149937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:01.149937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:01.149937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:01.149937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:01.149937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:01.149937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:01.149937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:01.149937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:01.151627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:01.151627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:01.151627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:01.151627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:01.151627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:01.151627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:01.151627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:01.151627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:01.151627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:01.151627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:01.151627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:01.151627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:01.151627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:01.152415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:01.153715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:01.154905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:01.154905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:01.154905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:01.154905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:01.154905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:01.154905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:03.802682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:03.804404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:03.804404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:03.804947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:03.805290 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:03.805545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:03.810455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:03.810964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:03.811431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:03.811855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:03.812443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:03.812443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:03.812443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:03.812443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.812443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:03.812443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:03.812443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.812443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:03.812443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.812443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:03.812443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:03.813571 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:03.813571 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:03.813571 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:50:03.814033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:03.814805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:03.817864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:03.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:03.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:03.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:03.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:03.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:03.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:50:03.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:03.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:03.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:03.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:03.818186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.820667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:03.820938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.820938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:03.822188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:03.823416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:03.824354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.824354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:03.824354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:03.824354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.824354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:03.824354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.824354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:03.824354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:03.824354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:03.824354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:03.826427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:03.826427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:03.826427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:03.826427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.826427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:03.826427 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.826427 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:03.826427 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:03.826427 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.826427 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:03.826427 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.826427 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:03.826652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:03.826917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:03.827167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:03.827167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:03.827167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:03.827167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:03.827167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.827167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:03.827167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:03.827167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:03.827167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.827167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:03.827167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:03.827167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:03.827167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:03.827423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:03.827423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:03.827423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:03.827423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:03.827423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.827423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:03.827423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:03.827423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.827423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:03.827423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.827423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:03.829896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:03.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:03.833782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.833782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:03.833782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.833782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:03.833782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:03.833782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:03.833782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:03.833782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:03.834038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:03.834443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:03.834823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:03.834823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:03.834823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.834823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:03.834823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:03.834823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.834823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:03.834823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.834823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:03.834823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.834823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:03.834823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:03.835149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:03.835652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:06.114025 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:06.114840 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:06.114840 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:06.115098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:06.116244 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:50:06.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:06.151161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:06.151825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:06.151825 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:06.151825 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:06.151825 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:06.151825 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:06.151825 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:06.151825 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:06.151825 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:06.152848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:06.152848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:06.152848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:06.152848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:06.152848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:06.152848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:06.152848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:06.152848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:06.152848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:06.152848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:06.152848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:06.152848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:06.153570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:06.153570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:06.153570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:06.153570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:06.153570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:06.153570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:06.153570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:06.153570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:06.153570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:06.153570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:06.153570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:06.153570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:06.153570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:06.154171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:06.154884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:06.155403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:06.155403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:06.155403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:06.155403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:06.155403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:06.155403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:08.803948 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:08.804951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:08.804951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:08.805492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:08.805871 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:08.806087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.807432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:08.807432 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.807432 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:08.807432 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:08.807432 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.807432 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:08.807432 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.807432 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:08.808216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.808216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:08.808216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:08.808216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:08.808216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:08.808216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:08.808216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:08.808216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.808216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:08.808216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:08.808216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:08.808216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.808509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:08.808509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:08.808509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:08.808509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:08.808509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:08.808509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:08.808509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:08.808509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:08.808509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.808509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:08.808509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:08.808509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.808509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:08.808819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.808819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:08.808969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:08.809359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.809805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:08.809805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.809805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:08.813317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:08.814028 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:08.814344 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:08.814729 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:50:08.815322 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:08.815622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.816633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:08.816633 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.816633 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:08.816633 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:08.816633 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.816633 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:08.816633 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.816633 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:08.818087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:08.818590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:08.818590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:08.818590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.818590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:08.818590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:08.818590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.818590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:08.818590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.818590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:08.819736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.819736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:08.819736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:08.819736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:08.819736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:08.819736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:08.819736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:08.819736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:50:08.819736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:08.819736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:08.819736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.819736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:08.820135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:08.820548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.820548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:08.820548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.820548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:08.821491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:08.821968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:08.822484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.822484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:08.822484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:08.822484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.822484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:08.822484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.822484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:08.822777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:08.823165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:08.823399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:08.823692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:08.824284 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:08.824623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.825315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:08.825315 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.825315 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:08.825315 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:08.825315 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.825315 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:08.825315 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.825315 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:08.826110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:08.826528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:08.826528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:08.826528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.826528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:08.826528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:08.826528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.826528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:08.826528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.826528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:08.827108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.827108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:08.827108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:08.827108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:08.827108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:08.827108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:08.827108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:08.827108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:08.827108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:08.827108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.827108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:08.827465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:08.827823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.827823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:08.827823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.827823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:08.828344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:08.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:08.829434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.829434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:08.829434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.829434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:08.829776 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:08.830081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:08.830200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:08.830383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:08.830867 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:08.831226 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:08.831459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.832234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:08.832234 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.832234 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:08.832234 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:08.832234 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.832234 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:08.832234 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.832234 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:08.833068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.833068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:08.833068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:08.833068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:08.833068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:08.833068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:08.833068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:08.833068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.833068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:08.833068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:08.833068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:08.833068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.833826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:08.833826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:08.833826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:08.833826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:08.833826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:08.833826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:08.833826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:08.833826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:08.833826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.833826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:08.833826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:08.833826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.833826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:08.834708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:08.835508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:08.836476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:08.836476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:08.836476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:08.836476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:08.836476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:08.836476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:11.114625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:11.115419 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:11.115419 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:11.115944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:11.148788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:11.149052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:11.149695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:11.149695 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:11.149695 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:11.149695 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:11.149695 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:11.149695 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:11.149695 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:11.149695 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:11.150563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:11.150563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:11.150563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:11.150563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:11.150563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:11.150563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:11.150563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:11.150563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:11.150563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:11.150563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:11.150563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:11.150563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:11.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:11.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:11.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:11.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:11.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:11.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:11.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:11.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:11.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:11.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:11.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:11.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:11.150941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:11.151429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:11.151823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:11.152371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:11.152371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:11.152371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:11.152371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:11.152371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:11.152371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:13.804610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:13.805784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:13.806014 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:13.807249 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:13.807590 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:13.807846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.808756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:13.808756 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.808756 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:13.808756 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:13.808756 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.808756 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:13.808756 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.808756 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:13.810543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.810543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:13.810543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:13.810543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:13.810543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:13.810543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:13.810543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:13.810543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.810543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:13.810543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:13.810543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:13.810543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.811073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:13.811073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:13.811073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:13.811073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:13.811073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:13.811073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:13.811073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:13.811073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:13.811073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.811073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:13.811073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:13.811073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.811073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:13.811672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.811672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:13.812477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:13.812893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.813407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:13.813407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.813407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:13.814040 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:13.814347 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:13.814478 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:13.814635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:50:13.814959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:13.815198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.815751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:13.815751 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.815751 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:13.815751 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:13.815751 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.815751 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:13.815751 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.815751 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:13.816782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:13.817895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:13.817895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:13.817895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.817895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:13.817895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:13.817895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.817895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:13.817895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.817895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:13.818676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.818676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:13.818676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:13.818676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:13.818676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:13.818676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:13.818676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:13.818676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:50:13.818676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:13.818676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:13.818676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.818676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:13.819356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:13.819906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.819906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:13.819906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.819906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:13.821410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:13.821982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:13.822641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.822641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:13.822641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:13.822641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.822641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:13.822641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.822641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:13.823145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:13.823772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:13.823772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:13.823772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:13.824040 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:13.824281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.827731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:13.828245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.828565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:13.828565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:13.828565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:13.828565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:13.828565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:13.828565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:13.828565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:13.828565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:13.828565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.828565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:13.828565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:13.828565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:13.828565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.828825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:13.828825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:13.828825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:13.828825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:13.828825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:13.828825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:13.828825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:13.828825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:13.828825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.828825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:13.828825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:13.828825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.828825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:13.829081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:13.829386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:13.829756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:13.829756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.829756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:13.829756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:13.829756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.829756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:13.829756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.829756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:13.830121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:13.830301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:13.830447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:13.830617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:13.831095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:13.831477 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:13.831688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.832315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:13.832315 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.832315 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:13.832315 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:13.832315 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.832315 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:13.832315 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.832315 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:13.833813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.833813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:13.833813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:13.833813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:13.833813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:13.833813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:13.833813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:13.833813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.833813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:13.833813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:13.833813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:13.833813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.834090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:13.834090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:13.834090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:13.834090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:13.834090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:13.834090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:13.834090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:13.834090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:13.834090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.834090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:13.834090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:13.834090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.834090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:13.834377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:13.834668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:13.835063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:13.835063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:13.835063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:13.835063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:13.835063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:13.835063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:16.115323 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:16.119536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:16.119536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:16.119536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:16.150858 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:16.151041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:16.156777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:16.157512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:16.157850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:16.158172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:16.159010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:16.159010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:16.159010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:16.159010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:16.159010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:16.159010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:16.159010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:16.159010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:16.159010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:18.805943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:18.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:18.806904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:18.807943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:18.808302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:18.808565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.809471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:18.809471 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.809471 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:18.809471 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:18.809471 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.809471 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:18.809471 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.809471 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:18.810393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.810393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:18.810393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:18.810393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:18.810393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:18.810393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:18.810393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:18.810393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.810393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:18.810393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:18.810393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:18.810393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.811048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:18.811048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:18.811048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:18.811048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:18.811048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:18.811048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:18.811048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:18.811048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:18.811048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.811048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:18.811048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:18.811048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.811048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:18.811774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:18.812572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.813753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:18.813753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:18.813753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.813753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:18.813753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.813753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:18.814556 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:18.815833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:18.816035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:18.816300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:50:18.816853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:18.817506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.818519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:18.818519 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.818519 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:18.818519 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:18.818519 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.818519 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:18.818519 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.818519 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:18.819933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:18.820612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:18.820612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:18.820612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.820612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:18.820612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:18.820612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.820612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:18.820612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.820612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:18.821442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.821442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:18.821442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:18.821442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:18.821442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:18.821442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:18.821442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:18.821442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:50:18.821442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:18.821442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:18.821442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.821442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:18.821852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:18.822277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.822277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:18.822277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.822277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:18.823244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:18.823953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:18.824790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.824790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:18.824790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:18.824790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.824790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:18.824790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.824790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:18.826025 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:18.826424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:18.826611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:18.826848 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:18.827191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:18.827429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.827891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:18.827891 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.827891 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:18.827891 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:18.827891 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.827891 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:18.827891 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.827891 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:18.828620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:18.829284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:18.829284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:18.829284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.829284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:18.829284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:18.829284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.829284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:18.829284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.829284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:18.829677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.829677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:18.829677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:18.829677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:18.829677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:18.829677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:18.829677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:18.829677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:18.829677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:18.829677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.829677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:18.830178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:18.831059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:18.831499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:18.832085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:18.832085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:18.832085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:18.832085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:18.832085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.832085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:18.832085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:18.832085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.832085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:18.832085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.832085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:18.832085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:18.832085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:18.832470 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:18.832470 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:18.832470 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:18.832720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:18.832942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.833871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:18.833871 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.833871 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:18.833871 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:18.833871 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.833871 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:18.833871 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.833871 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:18.834690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.834690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:18.834690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:18.834690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:18.834690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:18.834690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:18.834690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:18.834690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.834690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:18.834690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:18.834690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:18.834690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:18.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:18.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:18.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:18.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:18.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:18.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:18.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:18.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:18.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:18.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:18.835486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.835486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:18.835666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:18.836124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:18.836669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:18.836669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:18.836669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:21.117814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:21.118881 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:21.118881 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:21.118881 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:21.148441 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:21.148751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:21.149129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:21.149129 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:21.149129 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:21.149129 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:21.149129 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:21.149129 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:21.149129 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:21.149129 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:21.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:21.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:21.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:21.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:21.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:21.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:21.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:21.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:21.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:21.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:21.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:21.149798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:21.150231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:21.150231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:21.150231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:21.150231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:21.150231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:21.150231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:21.150231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:21.150231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:21.150231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:21.150231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:21.150231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:21.150231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:21.150231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:21.151362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:21.152593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:21.153196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:21.153196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:21.153196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:21.153196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:21.153196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:21.153196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:23.806913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:23.807797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:23.807797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:23.808699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:23.808882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:23.809001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.809662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:23.809662 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.809662 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:23.809662 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:23.809662 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.809662 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:23.809662 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.809662 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:23.811147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.811147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:23.811147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:23.811147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:23.811147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:23.811147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:23.811147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:23.811147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.811147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:23.811147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:23.811147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:23.811147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.811598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:23.811598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:23.811598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:23.811598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:23.811598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:23.811598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:23.811598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:23.811598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:23.811598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.811598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:23.811598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:23.811598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.811598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:23.812032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.812032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:23.813033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:23.813623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.814193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:23.814193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.814193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:23.815667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:23.816432 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:23.816432 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:23.816541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:50:23.816907 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:23.817104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.817731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:23.817731 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.817731 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:23.817731 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:23.817731 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.817731 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:23.817731 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.817731 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:23.818748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:23.819601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:23.820321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:23.820321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:23.820321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:50:23.820321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:23.820321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:23.820321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.820321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:23.820321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:23.820321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:23.820321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.820321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:23.820321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:23.820321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:23.820908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:23.820908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:23.820908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:23.820908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:23.820908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:23.820908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.820908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:23.820908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:23.820908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.820908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:23.820908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.820908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:23.821452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:23.822153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:23.823758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.823758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:23.823758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:23.823758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.823758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:23.823758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.823758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:23.826379 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:23.826567 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:23.826671 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:23.826809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:23.827175 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:23.827423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.827920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:23.827920 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.827920 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:23.827920 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:23.827920 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.827920 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:23.827920 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.827920 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:23.828714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:23.829185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:23.829185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:23.829185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.829185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:23.829185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:23.829185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.829185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:23.829185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.829185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:23.829628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.829628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:23.829628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:23.829628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:23.829628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:23.829628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:23.829628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:23.829628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:23.829628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:23.829628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.829628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:23.830016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:23.830722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:23.831156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:23.831706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:23.831706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:23.831706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:23.831706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:23.831706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.831706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:23.831706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:23.831706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.831706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:23.831706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.831706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:23.832022 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:23.832022 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:23.832022 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:23.832222 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:23.833514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:23.833761 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:23.833889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.834419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:23.834419 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.834419 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:23.834419 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:23.834419 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.834419 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:23.834419 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.834419 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:23.835221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.835221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:23.835221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:23.835221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:23.835221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:23.835221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:23.835221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:23.835221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.835221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:23.835221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:23.835221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:23.835221 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.835627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:23.835627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:23.835627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:23.835627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:23.835627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:23.835627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:23.835627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:23.835627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:23.835627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.835627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:23.835627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:23.835627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.835627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:23.835986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.835986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:23.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:23.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:23.837382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:23.837382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:23.837382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:26.118363 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:26.118973 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:26.119173 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:26.120634 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:26.153858 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:26.154187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:26.154878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:26.154878 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:26.154878 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:26.154878 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:26.154878 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:26.154878 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:26.154878 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:26.154878 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:26.155544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:26.155544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:26.155544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:26.155544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:26.155544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:26.155544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:26.155544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:26.155544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:26.155544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:26.155544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:26.155544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:26.155544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:26.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:26.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:26.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:26.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:26.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:26.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:26.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:26.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:26.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:26.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:26.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:26.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:26.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:26.156269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:26.156683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:26.157158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:26.157158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:26.157158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:26.157158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:26.157158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:26.157158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:28.807655 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:28.808289 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:28.808289 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:28.809820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:28.810350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:28.810654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.811912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:28.811912 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.811912 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:28.811912 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:28.811912 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.811912 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:28.811912 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.811912 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:28.813503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.813503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:28.813503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:28.813503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:28.813503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:28.813503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:28.813503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:28.813503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.813503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:28.813503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:28.813503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:28.813503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.813927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:28.813927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:28.813927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:28.813927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:28.813927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:28.813927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:28.813927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:28.813927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:28.813927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.813927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:28.813927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:28.813927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.813927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:28.814386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.814386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:28.815318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:28.815853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.816512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:28.816512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.816512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:28.816512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:28.816512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:28.816690 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:28.816778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:50:28.817269 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:28.817522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.818133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:28.818133 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.818133 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:28.818133 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:28.818133 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.818133 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:28.818133 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.818133 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:28.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:28.819812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:28.819812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:28.819812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.819812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:28.819812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:28.819812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.819812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:28.819812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.819812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:28.820600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.820600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:28.820600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:28.820600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:28.820600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:28.820600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:28.820600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:28.820600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:50:28.820600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:28.820600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:28.820600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.820600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:28.820967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:28.821360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.821360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:28.821360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.821360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:28.822117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:28.822563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:28.823194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.823194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:28.823194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:28.823194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.823194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:28.823194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.823194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:28.826618 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:28.826809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:28.826914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:28.827038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:28.827537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:28.827805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.828393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:28.828393 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.828393 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:28.828393 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:28.828393 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.828393 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:28.828393 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.828393 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:28.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:28.829588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:28.829588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:28.829588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.829588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:28.829588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:28.829588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.829588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:28.829588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.829588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:28.830055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.830055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:28.830055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:28.830055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:28.830055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:28.830055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:28.830055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:28.830055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:28.830055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:28.830055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.830055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:28.830401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:28.830814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.830814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:28.830814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.830814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:28.831169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:28.831660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:28.832184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.832184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:28.832184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.832184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:28.832184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:28.832184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:28.832395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:28.832439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:28.832808 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:28.833046 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:28.833223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.833808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:28.833808 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.833808 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:28.833808 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:28.833808 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.833808 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:28.833808 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.833808 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:28.834688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.834688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:28.834688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:28.834688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:28.834688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:28.834688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:28.834688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:28.834688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.834688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:28.834688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:28.834688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:28.834688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.835081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:28.835081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:28.835081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:28.835081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:28.835081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:28.835081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:28.835081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:28.835081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:28.835081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.835081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:28.835081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:28.835081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.835081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:28.835464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.835464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:28.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:28.836162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:28.836698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:28.836698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:28.836698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:31.119152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:31.119787 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:31.119837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:31.121170 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:31.152370 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:31.152805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:31.153757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:31.153757 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:31.153757 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:31.153757 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:31.153757 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:31.153757 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:31.153757 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:31.153757 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:31.154535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:31.154535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:31.154535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:31.154535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:31.154535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:31.154535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:31.154535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:31.154535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:31.154535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:31.154535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:31.154535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:31.154535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:31.155125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:31.155125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:31.155125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:31.155125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:31.155125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:31.155125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:31.155125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:31.155125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:31.155125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:31.155125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:31.155125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:31.155125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:31.155125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:31.155728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:31.156434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:31.157369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:31.157369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:31.157369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:31.157369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:31.157369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:31.157369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:33.808138 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:33.809116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:33.809116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:33.809706 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:33.809997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:33.810297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.811274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:33.811274 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.811274 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:33.811274 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:33.811274 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.811274 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:33.811274 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.811274 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:33.811971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.811971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:33.811971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:33.811971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:33.811971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:33.811971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:33.811971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:33.811971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.811971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:33.811971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:33.811971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:33.811971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.812615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:33.812615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:33.812615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:33.812615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:33.812615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:33.812615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:33.812615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:33.812615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:33.812615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.812615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:33.812615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:33.812615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.812615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:33.813248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:33.813968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.814917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:33.814917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:33.814917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.814917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:33.814917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.814917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:33.816578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:33.816797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:33.816866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:33.816968 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:50:33.817288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:33.817554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.817959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:33.817959 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.817959 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:33.817959 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:33.817959 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.817959 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:33.817959 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.817959 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:33.818815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:33.819350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:33.819350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:33.819350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.819350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:33.819350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:33.819350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.819350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:33.819350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.819350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:33.819641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.819641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:33.819641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:33.819641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:33.819641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:33.819641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:33.819641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:33.819641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:50:33.819641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:33.819641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:33.819641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.819641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:33.820009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:33.820420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:33.820823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:33.821369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:33.821369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:33.821369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:33.821369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:33.821369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:33.821369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.821369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:33.821369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:33.821369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.821369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:33.821369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.821369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:33.826963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:33.827320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:33.827434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:33.827568 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:33.828068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:33.828466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.829164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:33.829164 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.829164 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:33.829164 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:33.829164 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.829164 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:33.829164 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.829164 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:33.829978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:33.830412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:33.830412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:33.830412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.830412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:33.830412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:33.830412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.830412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:33.830412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.830412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:33.831180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.831180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:33.831180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:33.831180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:33.831180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:33.831180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:33.831180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:33.831180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:33.831180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:33.831180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.831180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:33.831528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:33.831909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.831909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:33.831909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.831909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:33.832288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:33.832768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:33.833292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.833292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:33.833292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.833292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:33.833292 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:33.833292 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:33.833292 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:33.833292 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:33.833555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:33.833774 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:33.833985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.834829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:33.834829 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.834829 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:33.834829 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:33.834829 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.834829 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:33.834829 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.834829 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:33.836301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.836301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:33.836301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:33.836301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:33.836301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:33.836301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:33.836301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:33.836301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.836301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:33.836301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:33.836301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:33.836301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.836637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:33.836637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:33.836637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:33.836637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:33.836637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:33.836637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:33.836637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:33.836637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:33.836637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.836637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:33.836637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:33.836637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.836637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:33.836971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.836971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:33.837931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:33.838355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:33.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:33.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:33.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:36.119879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:36.120556 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:36.120657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:36.122057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:36.156622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:36.157049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:36.157965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:36.157965 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:36.157965 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:36.157965 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:36.157965 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:36.157965 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:36.157965 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:36.157965 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:36.159113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:36.159113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:36.159113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:36.159113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:36.159113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:36.159113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:36.159113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:36.159113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:36.159113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:36.159113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:36.159113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:36.159113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:36.159835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:36.159835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:36.159835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:36.159835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:36.159835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:36.159835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:36.159835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:36.159835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:36.159835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:36.159835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:36.159835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:36.159835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:36.159835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:36.160806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:36.161727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:36.162623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:36.162623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:36.162623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:36.162623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:36.162623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:36.162623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:38.808792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:38.809751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:38.809751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:38.810562 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:38.810776 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:38.810936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.811590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:38.811590 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.811590 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:38.811590 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:38.811590 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.811590 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:38.811590 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.811590 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:38.813154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.813154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:38.813154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:38.813154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:38.813154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:38.813154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:38.813154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:38.813154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.813154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:38.813154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:38.813154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:38.813154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.813479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:38.813479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:38.813479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:38.813479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:38.813479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:38.813479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:38.813479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:38.813479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:38.813479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.813479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:38.813479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:38.813479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.813479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:38.813831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.813831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:38.814998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:38.815430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.815924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:38.815924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.815924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:38.816969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:38.817422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:38.817611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:38.817758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:50:38.818235 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:38.818411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.818880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:38.818880 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.818880 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:38.818880 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:38.818880 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.818880 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:38.818880 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.818880 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:38.819662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:38.820357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:38.820357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:38.820357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.820357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:38.820357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:38.820357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.820357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:38.820357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.820357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:38.820773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.820773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:38.820773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:38.820773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:38.820773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:38.820773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:38.820773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:38.820773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:50:38.820773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:38.820773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:38.820773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.820773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:38.821321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:38.821914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:38.822594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:38.823143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:38.823143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:38.823143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:38.823143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:38.823143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:38.823143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.823143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:38.823143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:38.823143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.823143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:38.823143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.823143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:38.827360 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:38.827607 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:38.827717 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:38.827844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:38.828321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:38.828590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.829161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:38.829161 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.829161 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:38.829161 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:38.829161 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.829161 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:38.829161 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.829161 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:38.829910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:38.830314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:38.830314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:38.830314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.830314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:38.830314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:38.830314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.830314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:38.830314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.830314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:38.830824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.830824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:38.830824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:38.830824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:38.830824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:38.830824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:38.830824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:38.830824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:38.830824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:38.830824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.830824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:38.831122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:38.831491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.831491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:38.831491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.831491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:38.831951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:38.832737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:38.833632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.833632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:38.833632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.833632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:38.833632 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:38.833632 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:38.833632 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:38.833632 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:38.834263 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:38.834521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:38.834749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.835402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:38.835402 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.835402 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:38.835402 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:38.835402 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.835402 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:38.835402 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.835402 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:38.836230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.836230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:38.836230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:38.836230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:38.836230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:38.836230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:38.836230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:38.836230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.836230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:38.836230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:38.836230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:38.836230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.836808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:38.836808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:38.836808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:38.836808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:38.836808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:38.836808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:38.836808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:38.836808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:38.836808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.836808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:38.836808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:38.836808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.836808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:38.837476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:38.838196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:38.839099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:38.839099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:38.839099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:38.839099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:38.839099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:38.839099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:41.120421 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:41.120978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:41.121174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:41.122532 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:41.153096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:41.153590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:41.154003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:41.154003 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:41.154003 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:41.154003 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:41.154003 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:41.154003 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:41.154003 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:41.154003 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:41.154736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:41.154736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:41.154736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:41.154736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:41.154736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:41.154736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:41.154736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:41.154736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:41.154736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:41.154736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:41.154736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:41.154736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:41.155339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:41.155339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:41.155339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:41.155339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:41.155339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:41.155339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:41.155339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:41.155339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:41.155339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:41.155339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:41.155339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:41.155339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:41.155339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:41.155923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:41.156612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:41.157484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:41.157484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:41.157484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:41.157484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:41.157484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:41.157484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:43.809509 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:43.810280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:43.810347 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:43.811516 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:43.811857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:43.812101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.813062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:43.813062 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.813062 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:43.813062 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:43.813062 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.813062 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:43.813062 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.813062 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:43.815119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.815119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:43.815119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:43.815119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:43.815119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:43.815119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:43.815119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:43.815119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.815119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:43.815119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:43.815119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:43.815119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.815588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:43.815588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:43.815588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:43.815588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:43.815588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:43.815588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:43.815588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:43.815588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:43.815588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.815588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:43.815588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:43.815588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.815588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:43.815978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.815978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:43.816480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:43.817665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.818683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:43.818683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.818683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:43.818683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:43.818683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:43.818683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:43.818868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:50:43.819467 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:43.819467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.819963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:43.819963 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.819963 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:43.819963 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:43.819963 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.819963 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:43.819963 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.819963 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:43.820680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:43.821503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:43.822376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:43.822376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:43.822376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:50:43.822376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:43.822376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:43.822376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.822376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:43.822376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:43.822376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:43.822376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.822376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:43.822376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:43.822376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:43.822987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:43.822987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:43.822987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:43.822987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:43.822987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:43.822987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.822987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:43.822987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:43.822987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.822987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:43.822987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.822987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:43.823512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:43.824189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:43.825011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.825011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:43.825011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:43.825011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.825011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:43.825011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.825011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:43.827735 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:43.827979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:43.828087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:43.828223 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:43.828677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:43.829040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.829777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:43.829777 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.829777 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:43.829777 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:43.829777 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.829777 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:43.829777 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.829777 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:43.830521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:43.830979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:43.830979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:43.830979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.830979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:43.830979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:43.830979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.830979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:43.830979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.830979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:43.831411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.831411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:43.831411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:43.831411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:43.831411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:43.831411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:43.831411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:43.831411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:43.831411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:43.831411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.831411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:43.831764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:43.832126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.832126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:43.832126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.832126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:43.832427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:43.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:43.833479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.833479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:43.833479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.833479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:43.833479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:43.833638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:43.833638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:43.833784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:43.834199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:43.834591 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:43.834874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.835565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:43.835565 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.835565 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:43.835565 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:43.835565 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.835565 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:43.835565 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.835565 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:43.836533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.836533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:43.836533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:43.836533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:43.836533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:43.836533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:43.836533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:43.836533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.836533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:43.836533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:43.836533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:43.836533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.836998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:43.836998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:43.836998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:43.836998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:43.836998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:43.836998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:43.836998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:43.836998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:43.836998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.836998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:43.836998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:43.836998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.836998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:43.837385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.837385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:43.837454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:43.837918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:43.838466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:43.838466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:43.838466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:45.771857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:45.772406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:45.772406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:45.779503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 07 15:50:45.779665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105503) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 07 15:50:45.779782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:50:45.822943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, resource={'id': '8b456c85-5e2d-47f8-9886-ebf65932cf32', 'name': 'private', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4a00c537-ed4f-405c-b241-82e7bc2cf3fa', '9fc89727-343d-403a-acd7-db6b4950e7b1'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:25:31Z', 'updated_at': '2026-05-07T15:25:34Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 42799}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.823161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.823401 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-b0f8152d-626b-440e-9383-919ec905ab38, external_ids_update=None, columns={'tag': 42799}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.823492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.824040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:45.824040 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:45.824040 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:45.824040 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:45.824040 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:45.824040 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:45.824040 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.824040 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:45.824914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.824914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:45.824914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:50:45.824914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:50:45.824914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:50:45.824914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:50:45.824914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:50:45.824914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:45.824914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:45.824914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:45.824914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:45.825219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:45.825504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:45.825504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.825504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:45.825504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:50:45.855485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, resource={'id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'name': 'public', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6aa26064-0a62-428b-a8ed-fef52fc73984', 'adf79775-4dfa-4110-bca0-59be9560299c'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-07T15:25:37Z', 'updated_at': '2026-05-07T15:25:45Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.855698 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.855987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-2b8b3cbc-130c-4894-94e1-ab0729ec7077, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.856160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.856750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:45.856750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:45.856750 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:45.856750 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:45.856750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:45.856750 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:45.856750 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.856750 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:45.858009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.858009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:45.858009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:50:45.858009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:50:45.858009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:50:45.858009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:50:45.858009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:50:45.858009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:45.858009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:45.858009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:45.858009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:45.858281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:45.858581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:45.858581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.858581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:45.858581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:50:45.890321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, resource={'id': '71aea053-d575-4d13-afc5-e6241ad6a15f', 'name': 'admin_net', 'tenant_id': '74d9c701d5cb4de6a15233057b01e413', 'project_id': '74d9c701d5cb4de6a15233057b01e413', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8e8a2cf7-97a7-4d62-ae4a-fa6c2139e811'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:30:51Z', 'updated_at': '2026-05-07T15:30:52Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 51150}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.890576 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.890867 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d5042c49-4c90-465a-b356-c18cf966a58b, external_ids_update=None, columns={'tag': 51150}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.890961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.891803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:45.891803 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:45.891803 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:45.891803 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:45.891803 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:45.891803 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:45.891803 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.891803 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:45.893051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.893051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:45.893051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:50:45.893051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:50:45.893051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:50:45.893051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:50:45.893051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:50:45.893051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:45.893051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:45.893051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:45.893051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:45.893418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:45.893708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:45.893708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.893708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:45.893708 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:50:45.925792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, resource={'id': '4fb571ad-ed2f-4edf-a199-cb52ff142698', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'tenant_id': '91e1e016e0f8434890282eb48ed58760', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2e6720c7-3159-41b9-80a2-fa6cf6a18808'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:31:26Z', 'updated_at': '2026-05-07T15:31:27Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 29129}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.926230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.926902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13b8a284-1038-44ba-86e4-667f771dfae2, external_ids_update=None, columns={'tag': 29129}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.927118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.928124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:45.928124 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:45.928124 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:45.928124 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:45.928124 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:45.928124 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:45.928124 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.928124 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:45.929382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.929382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:45.929382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:50:45.929382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:50:45.929382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:50:45.929382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:50:45.929382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:50:45.929382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:45.929382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:45.929382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:45.929382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:45.929630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:45.929895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:45.929895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:45.929895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:45.929895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:50:45.998979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, resource={'id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'enable_snat': True}, 'gw_port_id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-jseci', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'ip_address': '172.24.5.213', 'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984'}, {'ip_address': '2001:db8::15a', 'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 3, 'tenant_id': '91e1e016e0f8434890282eb48ed58760'}, resource_type=routers, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.999405 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.999743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:45.999963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:46.000191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.001118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:46.001118 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:46.001118 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:46.001118 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:46.001118 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:46.001118 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:46.001118 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.001118 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:46.001371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Unable to update router 0938bfb8-1052-49e2-bd69-e5fda45e8a2f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.002991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.002991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:46.002991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:50:46.002991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:50:46.002991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:50:46.002991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:50:46.002991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 07 15:50:46.002991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 07 15:50:46.002991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 07 15:50:46.002991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 07 15:50:46.002991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:46.003245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:46.003506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:46.003506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:46.003506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:46.003506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:46.003506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:46.003506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:46.003506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:46.003506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.003506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:46.003506 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:50:46.104845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, resource={'id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'name': '', 'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'project_id': '', 'mac_address': 'fa:16:3e:f6:db:65', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000172501', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:46.105166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '6aa26064-0a62-428b-a8ed-fef52fc73984 adf79775-4dfa-4110-bca0-59be9560299c', 'neutron:network_name': 'neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.213/24', '2001:db8::15a/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:46.105530 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=b30ffafe-8df3-4064-bcbf-e3d1daad619c, lrouter_port=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:46.105784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.106521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:46.106521 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:46.106521 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:46.106521 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:46.106521 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:46.106521 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:46.106521 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.106521 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:46.107350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.107350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:46.107350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:50:46.107350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:50:46.107350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:50:46.107350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:50:46.107350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 07 15:50:46.107350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:46.107350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:46.107350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:46.107350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:46.107602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:46.107846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:46.107846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.107846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:46.107846 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_for_inconsistencies finished in 0.335 seconds
May 07 15:50:46.120898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:46.121236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:46.121423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:46.122627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:46.156638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:46.157006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.157916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:46.157916 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:46.157916 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:46.157916 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:46.157916 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:46.157916 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:46.157916 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.157916 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:46.158954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.158954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:46.158954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:46.158954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:46.158954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:46.158954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:46.158954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:46.158954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:46.158954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:46.158954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:46.158954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:46.158954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:46.159259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:46.159259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:46.159259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:46.159259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:46.159259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:46.159259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:46.159259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:46.159259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:46.159259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:46.159259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:46.159259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:46.159259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:46.159259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:46.159504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.159504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:46.160067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:46.160661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:46.160992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:46.160992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:46.160992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:48.812400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:48.812967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:48.812967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:48.813394 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:48.813591 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:48.813716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.814198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:48.814198 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.814198 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:48.814198 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:48.814198 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.814198 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:48.814198 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.814198 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:48.815001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.815001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:48.815001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:48.815001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:48.815001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:48.815001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:48.815001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:48.815001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.815001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:48.815001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:48.815001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:48.815001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.815476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:48.815476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:48.815476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:48.815476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:48.815476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:48.815476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:48.815476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:48.815476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:48.815476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.815476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:48.815476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:48.815476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.815476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:48.815829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:48.816272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.816793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:48.816793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:48.816793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.816793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:48.816793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.816793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:48.819179 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:48.819452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:48.819539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:48.819627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:50:48.820108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:48.820294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.820747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:48.820747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.820747 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:48.820747 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:48.820747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.820747 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:48.820747 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.820747 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:48.821414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:48.821867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:48.821867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:48.821867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.821867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:48.821867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:48.821867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.821867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:48.821867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.821867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:48.822181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.822181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:48.822181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:48.822181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:48.822181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:48.822181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:48.822181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:48.822181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:50:48.822181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:48.822181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:48.822181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.822181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:48.822586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:48.822998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:48.823471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:48.824008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:48.824008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:48.824008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:48.824008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:48.824008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:48.824008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.824008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:48.824008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:48.824008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.824008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:48.824008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.824008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:48.828031 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:48.828221 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:48.828309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:48.828409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:48.828946 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:48.829157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.829661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:48.829661 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.829661 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:48.829661 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:48.829661 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.829661 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:48.829661 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.829661 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:48.830300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:48.830762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:48.830762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:48.830762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.830762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:48.830762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:48.830762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.830762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:48.830762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.830762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:48.831024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.831024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:48.831024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:48.831024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:48.831024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:48.831024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:48.831024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:48.831024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:48.831024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:48.831024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.831024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:48.831371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:48.831781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:48.832201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:48.832778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:48.832778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:48.832778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:48.832778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:48.832778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.832778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:48.832778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:48.832778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.832778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:48.832778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.832778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:48.833596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:48.833758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:48.833829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:48.833963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:48.834337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:48.834528 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:48.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.835100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:48.835100 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.835100 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:48.835100 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:48.835100 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.835100 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:48.835100 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.835100 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:48.835790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.835790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:48.835790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:48.835790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:48.835790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:48.835790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:48.835790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:48.835790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.835790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:48.835790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:48.835790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:48.835790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.836232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:48.836232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:48.836232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:48.836232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:48.836232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:48.836232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:48.836232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:48.836232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:48.836232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.836232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:48.836232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:48.836232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.836232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:48.836623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:48.837066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:48.837640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:48.837640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:48.837640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:48.837640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:48.837640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:48.837640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:51.122239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:51.122239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:51.122239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:51.123106 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:51.156251 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:51.156480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:51.157015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:51.157015 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:51.157015 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:51.157015 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:51.157015 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:51.157015 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:51.157015 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:51.157015 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:51.157761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:51.157761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:51.157761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:51.157761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:51.157761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:51.157761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:51.157761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:51.157761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:51.157761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:51.157761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:51.157761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:51.157761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:51.158553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:51.158553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:51.158553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:51.158553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:51.158553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:51.158553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:51.158553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:51.158553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:51.158553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:51.158553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:51.158553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:51.158553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:51.158553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:51.159134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:51.160044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:51.160918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:51.160918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:51.160918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:51.160918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:51.160918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:51.160918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:53.812812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:53.813498 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:53.813498 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:53.815325 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:53.815325 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:53.815325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.815325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:53.815325 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.815325 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:53.815325 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:53.815325 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.815325 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:53.815325 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.815325 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:53.816403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.816403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:53.816403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:53.816403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:53.816403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:53.816403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:53.816403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:53.816403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.816403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:53.816403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:53.816403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:53.816403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.817130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:53.817130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:53.817130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:53.817130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:53.817130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:53.817130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:53.817130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:53.817130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:53.817130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.817130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:53.817130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:53.817130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.817130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:53.817523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:53.817985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.818549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:53.818549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:53.818549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.818549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:53.818549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.818549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:53.819512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:53.819715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:53.819832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:53.819954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:50:53.820437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:53.821276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:53.821761 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.821761 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:53.821761 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:53.821761 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.821761 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:53.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:53.822567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:53.822934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:53.822934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:53.822934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.822934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:53.822934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:53.822934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.822934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:53.822934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.822934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:53.823406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.823406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:53.823406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:53.823406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:53.823406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:53.823406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:53.823406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:53.823406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:50:53.823406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:53.823406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:53.823406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.823406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:53.823674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:53.823949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.823949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:53.823949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.823949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:53.824491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:53.824817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:53.825196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.825196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:53.825196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:53.825196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.825196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:53.825196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.825196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:53.828417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:53.829780 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:53.829904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:53.830039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:53.830692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:53.830692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.831406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:53.831406 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.831406 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:53.831406 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:53.831406 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.831406 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:53.831406 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.831406 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:53.832266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:53.832706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:53.832706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:53.832706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.832706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:53.832706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:53.832706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.832706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:53.832706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.832706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:53.833222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.833222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:53.833222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:53.833222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:53.833222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:53.833222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:53.833222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:53.833222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:53.833222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:53.833222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.833222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:53.833623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:53.834035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.834035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:53.834035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.834035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:53.834519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:53.835057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:53.835629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.835629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:53.835629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.835629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:53.835629 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:53.835629 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:53.835629 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:53.835629 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:53.836865 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:53.836865 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:53.836865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.836993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:53.836993 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.836993 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:53.836993 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:53.836993 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.836993 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:53.836993 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.836993 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:53.837575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.837575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:53.837575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:53.837575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:53.837575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:53.837575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:53.837575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:53.837575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.837575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:53.837575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:53.837575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:53.837575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.837962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:53.837962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:53.837962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:53.837962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:53.837962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:53.837962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:53.837962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:53.837962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:53.837962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.837962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:53.837962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:53.837962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.837962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:53.838373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:53.838826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:53.839405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:53.839405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:53.839405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:53.839405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:53.839405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:53.839405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:56.121906 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:56.122485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:56.122485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:56.123346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:56.161426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:56.161426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:56.161693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:56.161693 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:56.161693 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:56.161693 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:56.161693 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:56.161693 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:56.161693 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:56.161693 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:56.162636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:56.162636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:56.162636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:56.162636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:56.162636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:56.162636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:56.162636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:56.162636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:56.162636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:56.162636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:56.162636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:56.162636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:56.162982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:56.162982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:56.162982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:56.162982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:56.162982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:56.162982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:56.162982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:56.162982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:56.162982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:56.162982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:56.162982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:56.162982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:56.162982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:56.163335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:56.163335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:56.163485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:56.163893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:56.164419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:56.164419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:56.164419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:58.813497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:58.813945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:58.814082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:58.815512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:58.815512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:58.815512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.815768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:58.815768 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.815768 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:58.815768 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:58.815768 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.815768 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:58.815768 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.815768 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:58.817224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.817224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:58.817224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:58.817224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:58.817224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:58.817224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:58.817224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:58.817224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.817224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:58.817224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:58.817224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:58.817224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.817467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:58.817467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:58.817467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:58.817467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:58.817467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:58.817467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:58.817467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:58.817467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:58.817467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.817467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:58.817467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:58.817467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.817467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:58.817755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.817755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:58.818780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:58.819124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.819494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:58.819494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.819494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:58.819783 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:58.819996 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:58.820105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:58.820238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:50:58.820793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:58.820793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.820905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:58.820905 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.820905 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:58.820905 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:58.820905 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.820905 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:58.820905 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.820905 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:58.821761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:58.822053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:58.822053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:58.822053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.822053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:58.822053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:58.822053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.822053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:58.822053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.822053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:58.823536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.823536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:58.823536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:58.823536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:58.823536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:58.823536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:58.823536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:58.823536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:50:58.823536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:58.823536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:58.823536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.823536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:58.823812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:58.824068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:58.824347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:58.824686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:58.824686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:58.824686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:58.824686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:58.824686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:58.824686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.824686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:58.824686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:58.824686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.824686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:58.824686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.824686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:58.829792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:58.830009 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:58.830113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:58.830262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:58.831121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:58.831121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.831121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:58.831121 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.831121 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:58.831121 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:58.831121 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.831121 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:58.831121 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.831121 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:50:58.831771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:50:58.832039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:58.832039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:50:58.832039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.832039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:50:58.832039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:50:58.832039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.832039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:50:58.832039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.832039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:50:58.832678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.832678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:58.832678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:58.832678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:58.832678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:58.832678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:58.832678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:58.832678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:58.832678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:58.832678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.832678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:58.832920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:58.833166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.833166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:58.833166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.833166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:58.833794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:58.834139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:58.834496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.834496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:58.834496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.834496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:50:58.834877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:50:58.835059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:50:58.835168 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:50:58.835342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:50:58.835530 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:58.835737 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:50:58.835891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.836289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:50:58.836289 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.836289 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:50:58.836289 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:50:58.836289 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.836289 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:50:58.836289 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.836289 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:50:58.837084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.837084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:50:58.837084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:58.837084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:50:58.837084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:50:58.837084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:58.837084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:58.837084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.837084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:58.837084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:58.837084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:50:58.837084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.837343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:50:58.837343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:58.837343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:50:58.837343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:58.837343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:50:58.837343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:50:58.837343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:58.837343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:50:58.837343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.837343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:50:58.837343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:50:58.837343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.837343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:50:58.837609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.837609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:50:58.838097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:50:58.838413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:50:58.838764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:50:58.838764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:50:58.838764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:01.122539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:01.123069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:01.123120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:01.124304 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:01.160499 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:01.160878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:01.161837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:01.161837 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:01.161837 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:01.161837 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:01.161837 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:01.161837 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:01.161837 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:01.161837 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:01.162744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:01.162744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:01.162744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:01.162744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:01.162744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:01.162744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:01.162744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:01.162744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:01.162744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:01.162744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:01.162744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:01.162744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:01.163419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:01.163419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:01.163419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:01.163419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:01.163419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:01.163419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:01.163419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:01.163419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:01.163419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:01.163419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:01.163419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:01.163419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:01.163419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:01.164061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:01.165008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:01.165880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:01.165880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:01.165880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:01.165880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:01.165880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:01.165880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:03.814120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:03.814795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:03.814893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:03.815842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:03.816156 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:03.816414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.817248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:03.817248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.817248 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:03.817248 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:03.817248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.817248 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:03.817248 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.817248 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:03.818236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.818236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:03.818236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:03.818236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:03.818236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:03.818236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:03.818236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:03.818236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.818236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:03.818236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:03.818236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:03.818236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.818638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:03.818638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:03.818638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:03.818638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:03.818638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:03.818638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:03.818638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:03.818638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:03.818638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.818638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:03.818638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:03.818638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.818638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:03.819047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.819047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:03.819336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:03.819714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:03.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.820077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:03.820283 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:03.820678 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:03.820833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:03.820950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:51:03.821413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:03.821718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.822752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:03.822752 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.822752 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:03.822752 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:03.822752 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.822752 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:03.822752 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.822752 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:03.823530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:03.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:03.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:03.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:03.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:03.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:03.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:03.824537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.824537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:03.824537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:03.824537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:03.824537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:03.824537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:03.824537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:03.824537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:51:03.824537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:03.824537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:03.824537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.824537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:03.824826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:03.825099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.825099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:03.825099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.825099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:03.825527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:03.825904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:03.826297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.826297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:03.826297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:03.826297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.826297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:03.826297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.826297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:03.830339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:03.830661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:03.830769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:03.830929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:03.831410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:03.831673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.834527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:03.834527 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.834527 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:03.834527 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:03.834527 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.834527 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:03.834527 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.834527 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:03.835365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:03.835673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:03.835673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:03.835673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.835673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:03.835673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:03.835673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.835673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:03.835673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.835673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:03.836258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.836258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:03.836258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:03.836258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:03.836258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:03.836258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:03.836258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:03.836258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:03.836258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:03.836258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.836258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:03.836540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:03.836818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.836818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:03.836818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.836818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:03.837528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:03.837901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:03.838270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.838270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:03.838270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.838270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:03.838270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:03.838399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:03.838483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:03.838659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:03.839190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:03.839693 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:03.839948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.840686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:03.840686 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.840686 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:03.840686 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:03.840686 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.840686 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:03.840686 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.840686 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:03.841630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.841630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:03.841630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:03.841630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:03.841630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:03.841630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:03.841630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:03.841630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.841630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:03.841630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:03.841630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:03.841630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.841908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:03.841908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:03.841908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:03.841908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:03.841908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:03.841908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:03.841908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:03.841908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:03.841908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.841908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:03.841908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:03.841908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.841908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:03.842165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.842165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:03.842594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:03.842940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:03.843311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:03.843311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:03.843311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:06.123128 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:06.123833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:06.123833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:06.124477 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:06.126336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=434 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:51:06.154045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:06.154477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:06.155291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:06.155291 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:06.155291 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:06.155291 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:06.155291 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:06.155291 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:06.155291 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:06.155291 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:06.155999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:06.155999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:06.155999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:06.155999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:06.155999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:06.155999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:06.155999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:06.155999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:06.155999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:06.155999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:06.155999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:06.155999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:06.156680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:06.156680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:06.156680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:06.156680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:06.156680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:06.156680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:06.156680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:06.156680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:06.156680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:06.156680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:06.156680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:06.156680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:06.156680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:06.158445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:06.158882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:06.159329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:06.159329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:06.159329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:06.159329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:06.159329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:06.159329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:08.815014 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:08.815837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:08.815837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:08.817425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:08.817425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:08.817425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.817976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:08.817976 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.817976 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:08.817976 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:08.817976 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.817976 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:08.817976 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.817976 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:08.818989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.818989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:08.818989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:08.818989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:08.818989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:08.818989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:08.818989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:08.818989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.818989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:08.818989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:08.818989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:08.818989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.819402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:08.819402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:08.819402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:08.819402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:08.819402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:08.819402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:08.819402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:08.819402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:08.819402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.819402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:08.819402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:08.819402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.819402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:08.819749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.819749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:08.820024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:08.820529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.821155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:08.821155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.821155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:08.821155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:08.821155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:08.821155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:08.821155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:51:08.821426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:08.821641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.822144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:08.822144 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.822144 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:08.822144 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:08.822144 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.822144 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:08.822144 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.822144 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:08.824164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:08.824704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:08.824704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:08.824704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.824704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:08.824704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:08.824704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.824704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:08.824704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.824704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:08.824996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.824996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:08.824996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:08.824996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:08.824996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:08.824996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:08.824996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:08.824996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:51:08.824996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:08.824996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:08.824996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.824996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:08.825400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:08.825811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.825811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:08.825811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.825811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:08.826092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:08.826583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:08.827106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.827106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:08.827106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:08.827106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.827106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:08.827106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.827106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:08.830884 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:08.831547 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:08.831735 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:08.831972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:08.832471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:08.832749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.833516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:08.833516 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.833516 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:08.833516 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:08.833516 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.833516 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:08.833516 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.833516 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:08.834765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:08.835272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:08.835272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:08.835272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.835272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:08.835272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:08.835272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.835272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:08.835272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.835272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:08.836163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.836163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:08.836163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:08.836163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:08.836163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:08.836163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:08.836163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:08.836163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:08.836163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:08.836163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.836163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:08.836538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:08.836902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.836902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:08.836902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.836902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:08.837427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:08.837952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:08.838482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.838482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:08.838482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.838482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:08.838482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:08.838639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:08.838747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:08.838904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:08.839204 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:08.839429 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:08.839565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.839994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:08.839994 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.839994 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:08.839994 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:08.839994 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.839994 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:08.839994 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.839994 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:08.840687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.840687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:08.840687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:08.840687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:08.840687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:08.840687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:08.840687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:08.840687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.840687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:08.840687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:08.840687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:08.840687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.841070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:08.841070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:08.841070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:08.841070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:08.841070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:08.841070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:08.841070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:08.841070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:08.841070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.841070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:08.841070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:08.841070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.841070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:08.841460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:08.841906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:08.842453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:08.842453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:08.842453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:08.842453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:08.842453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:08.842453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:11.123693 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:11.124149 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:11.124483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:11.125793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:11.163099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:11.163507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:11.164373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:11.164373 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:11.164373 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:11.164373 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:11.164373 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:11.164373 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:11.164373 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:11.164373 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:11.165433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:11.165433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:11.165433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:11.165433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:11.165433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:11.165433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:11.165433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:11.165433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:11.165433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:11.165433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:11.165433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:11.165433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:11.166088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:11.166088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:11.166088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:11.166088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:11.166088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:11.166088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:11.166088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:11.166088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:11.166088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:11.166088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:11.166088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:11.166088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:11.166088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:11.166773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:11.167665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:11.168309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:11.168309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:11.168309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:11.168309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:11.168309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:11.168309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:13.815882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:13.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:13.817365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:13.817768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:13.818002 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:13.818090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:13.821075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:13.821642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:13.821996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:13.822609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:13.823293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:13.823293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:13.823293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:13.823293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.823293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:13.823293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:13.823293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.823293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:13.823293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.823293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:13.823293 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:13.823293 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:13.823624 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:13.823624 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:51:13.823624 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:13.823624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.823838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:13.823838 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.823838 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:13.823838 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:13.823838 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.823838 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:13.823838 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.823838 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:13.824822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:13.825321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:13.825321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:13.825321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.825321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:13.825321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:13.825321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.825321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:13.825321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.825321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:13.826494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.826494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:13.826494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:13.826494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:13.826494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:13.826494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:13.826494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:13.826494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:51:13.826494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:13.826494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:13.826494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.826494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:13.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:13.827195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.827195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:13.827195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.827195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:13.828239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:13.828666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:13.829198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.829198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:13.829198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:13.829198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.829198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:13.829198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.829198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:13.833418 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:13.833668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:13.834046 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:13.834046 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:13.834275 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:13.834501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.834845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:13.834845 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.834845 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:13.834845 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:13.834845 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.834845 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:13.834845 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.834845 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:13.839063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:13.840092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:13.840838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:13.840838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:13.840838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:13.840838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:13.840838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.840838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:13.840838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:13.840838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:13.840838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.840838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:13.840838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:13.840838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:13.840838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:13.841505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:13.841505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:13.841505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:13.841505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:13.841505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.841505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:13.841505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:13.841505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.841505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:13.841505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.841505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:13.841505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:13.842069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.842907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:13.843894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:13.843894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:13.843894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:13.843894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:13.843894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:13.843894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:13.844228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:13.844967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:13.845580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.845580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:13.845580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:13.845580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.845580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:13.845580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.845580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:13.845580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.845580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:13.845580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:13.845580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:13.845580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:13.845580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:13.846022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:13.846581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:13.846581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:13.846581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:13.846581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:13.846581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:13.846581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:13.846581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:13.846581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:13.846581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:13.846581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:13.846581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:13.846581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:16.124381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:16.124879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:16.125128 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:16.126618 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:16.159388 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:16.159791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:16.160629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:16.160629 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:16.160629 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:16.160629 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:16.160629 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:16.160629 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:16.160629 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:16.160629 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:16.161419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:16.161419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:16.161419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:16.161419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:16.161419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:16.161419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:16.161419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:16.161419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:16.161419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:16.161419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:16.161419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:16.161419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:16.161761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:16.161761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:16.161761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:16.161761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:16.161761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:16.161761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:16.161761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:16.161761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:16.161761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:16.161761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:16.161761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:16.161761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:16.161761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:16.162081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:16.162081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:16.162164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:16.162620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:16.163095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:16.163095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:16.163095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:18.816986 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:18.817781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:18.817781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:18.818975 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:18.819229 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:18.819413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.820099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:18.820099 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.820099 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:18.820099 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:18.820099 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.820099 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:18.820099 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.820099 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:18.821613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.821613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:18.821613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:18.821613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:18.821613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:18.821613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:18.821613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:18.821613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.821613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:18.821613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:18.821613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:18.821613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.822334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:18.822334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:18.822334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:18.822334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:18.822334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:18.822334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:18.822334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:18.822334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:18.822334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.822334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:18.822334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:18.822334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.822334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:18.822901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.822901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:18.823273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:18.823820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.824606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:18.824606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.824606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:18.824606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:18.824606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:18.824606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:18.824606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:51:18.824819 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:18.825273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.825856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:18.825856 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.825856 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:18.825856 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:18.825856 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.825856 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:18.825856 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.825856 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:18.827002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:18.827530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:18.827530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:18.827530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.827530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:18.827530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:18.827530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.827530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:18.827530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.827530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:18.827827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.827827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:18.827827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:18.827827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:18.827827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:18.827827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:18.827827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:18.827827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:51:18.827827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:18.827827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:18.827827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.827827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:18.828191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:18.828615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.828615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:18.828615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.828615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:18.828855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:18.829364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:18.829857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.829857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:18.829857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:18.829857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.829857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:18.829857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.829857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:18.833968 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:18.834374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:18.834488 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:18.834613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:18.834962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:18.835398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.836116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:18.836116 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.836116 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:18.836116 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:18.836116 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.836116 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:18.836116 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.836116 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:18.836893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:18.837319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:18.837319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:18.837319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.837319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:18.837319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:18.837319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.837319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:18.837319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.837319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:18.837778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.837778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:18.837778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:18.837778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:18.837778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:18.837778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:18.837778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:18.837778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:18.837778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:18.837778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.837778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:18.838131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:18.838480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.838480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:18.838480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.838480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:18.838894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:18.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:18.840086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.840086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:18.840086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.840086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:18.840086 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:18.840086 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:18.840086 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:18.840086 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:18.840369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:18.840764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:18.840999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.841740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:18.841740 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.841740 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:18.841740 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:18.841740 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.841740 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:18.841740 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.841740 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:18.842631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.842631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:18.842631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:18.842631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:18.842631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:18.842631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:18.842631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:18.842631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.842631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:18.842631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:18.842631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:18.842631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.843021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:18.843021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:18.843021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:18.843021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:18.843021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:18.843021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:18.843021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:18.843021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:18.843021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.843021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:18.843021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:18.843021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.843021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:18.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:18.843641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:18.844090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:18.844611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:18.844611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:18.844611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:21.124893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:21.125446 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:21.125809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:21.126823 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:21.156946 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:21.157332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:21.158046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:21.158046 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:21.158046 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:21.158046 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:21.158046 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:21.158046 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:21.158046 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:21.158046 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:21.158719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:21.158719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:21.158719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:21.158719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:21.158719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:21.158719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:21.158719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:21.158719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:21.158719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:21.158719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:21.158719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:21.158719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:21.159417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:21.159417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:21.159417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:21.159417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:21.159417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:21.159417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:21.159417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:21.159417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:21.159417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:21.159417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:21.159417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:21.159417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:21.159417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:21.159934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:21.161294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:21.162445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:21.162445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:21.162445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:21.162445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:21.162445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:21.162445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:23.817696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:23.818746 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:23.818944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:23.819844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:23.820294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:23.820484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.821326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:23.821326 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.821326 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:23.821326 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:23.821326 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.821326 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:23.821326 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.821326 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:23.822432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.822432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:23.822432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:23.822432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:23.822432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:23.822432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:23.822432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:23.822432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.822432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:23.822432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:23.822432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:23.822432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.823634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:23.823634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:23.823634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:23.823634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:23.823634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:23.823634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:23.823634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:23.823634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:23.823634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.823634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:23.823634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:23.823634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.823634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:23.824641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:23.825831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.827306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:23.827306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:23.827306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.827306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:23.827306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.827306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:23.827306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:23.827306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:23.827306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:23.827306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:23.827633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.828011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:23.828577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.828577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:23.828577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.828577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:23.828577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:23.828577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:23.828577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:23.828577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:23.828577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:23.828577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:51:23.828577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:23.828577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:23.828934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:23.829320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.830218 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:23.834652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:23.834786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:23.834786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:23.834908 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:23.835229 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:23.835431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.836007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:23.836007 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.836007 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:23.836007 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:23.836007 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.836007 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:23.836007 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.836007 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:23.837403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:23.838490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:23.839350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:23.839350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:23.839350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:23.839350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:23.839350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.839350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:23.839350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:23.839350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:23.839350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.839350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:23.839350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:23.839350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:23.839350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:23.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:23.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:23.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:23.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:23.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:23.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:23.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:23.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:23.841111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:23.841952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:23.843169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.843169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:23.843169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.843169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:23.843169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:23.843169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:23.843169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:23.843169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:23.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:23.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:23.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:23.843504 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.843504 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:23.843504 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:23.843504 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.843504 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:23.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:23.844025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.844025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:23.844025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:23.844025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:23.844025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:23.844025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:23.844025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:23.844025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.844025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:23.844025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:23.844025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:23.844025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.844433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:23.844433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:23.844433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:23.844433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:23.844433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:23.844433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:23.844433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:23.844433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:23.844433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.844433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:23.844433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:23.844433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.844433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:23.844757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.844757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:23.845609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:23.846076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:23.846615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:23.846615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:23.846615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:26.125768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:26.126647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:26.126647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:26.127974 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:26.165076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:26.165425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:26.166341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:26.166341 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:26.166341 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:26.166341 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:26.166341 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:26.166341 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:26.166341 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:26.166341 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:26.167842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:26.167842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:26.167842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:26.167842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:26.167842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:26.167842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:26.167842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:26.167842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:26.167842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:26.167842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:26.167842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:26.167842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:26.168231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:26.168231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:26.168231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:26.168231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:26.168231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:26.168231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:26.168231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:26.168231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:26.168231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:26.168231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:26.168231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:26.168231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:26.168231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:26.168546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:26.168546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:26.169622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:26.170067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:26.170559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:26.170559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:26.170559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:28.818441 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:28.819273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:28.819273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:28.820184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:28.820561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:28.820801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.821685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:28.821685 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.821685 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:28.821685 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:28.821685 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.821685 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:28.821685 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.821685 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:28.823156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.823156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:28.823156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:28.823156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:28.823156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:28.823156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:28.823156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:28.823156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.823156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:28.823156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:28.823156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:28.823156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.823831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:28.823831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:28.823831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:28.823831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:28.823831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:28.823831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:28.823831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:28.823831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:28.823831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.823831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:28.823831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:28.823831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.823831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:28.824511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.824511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:28.824833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:28.825713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.826699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:28.826699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.826699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:28.826699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:28.826699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:28.826699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:28.826699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:51:28.826699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:28.827140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.827635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:28.827635 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.827635 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:28.827635 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:28.827635 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.827635 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:28.827635 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.827635 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:28.828606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:28.829200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:28.829200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:28.829200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.829200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:28.829200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:28.829200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.829200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:28.829200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.829200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:28.829451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.829451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:28.829451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:28.829451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:28.829451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:28.829451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:28.829451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:28.829451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:51:28.829451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:28.829451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:28.829451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.829451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:28.829727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:28.830101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.830101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:28.830101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.830101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:28.830622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:28.831286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:28.832180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.832180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:28.832180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:28.832180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.832180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:28.832180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.832180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:28.834754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:28.834993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:28.835101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:28.835242 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:28.835699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:28.836046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:28.836746 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.836746 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:28.836746 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:28.836746 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.836746 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:28.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.836746 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:28.837497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:28.837985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:28.837985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:28.837985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.837985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:28.837985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:28.837985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.837985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:28.837985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.837985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:28.838432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.838432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:28.838432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:28.838432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:28.838432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:28.838432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:28.838432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:28.838432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:28.838432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:28.838432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.838432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:28.838990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:28.839645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:28.839954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:28.840402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:28.840402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:28.840402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:28.840402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:28.840402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.840402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:28.840402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:28.840402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.840402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:28.840402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.840402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:28.840645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:28.840983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:28.840983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:28.841104 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:28.841531 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:28.841907 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:28.842127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.842866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:28.842866 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.842866 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:28.842866 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:28.842866 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.842866 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:28.842866 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.842866 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:28.843717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.843717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:28.843717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:28.843717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:28.843717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:28.843717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:28.843717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:28.843717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.843717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:28.843717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:28.843717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:28.843717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.844338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:28.844338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:28.844338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:28.844338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:28.844338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:28.844338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:28.844338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:28.844338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:28.844338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.844338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:28.844338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:28.844338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.844338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:28.844981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:28.845619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:28.846099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:28.846099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:28.846099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:28.846099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:28.846099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:28.846099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:31.126835 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:31.127945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:31.127945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:31.129072 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:31.162349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:31.162551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:31.163166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:31.163166 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:31.163166 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:31.163166 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:31.163166 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:31.163166 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:31.163166 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:31.163166 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:31.163885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:31.163885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:31.163885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:31.163885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:31.163885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:31.163885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:31.163885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:31.163885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:31.163885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:31.163885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:31.163885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:31.163885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:31.164268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:31.164268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:31.164268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:31.164268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:31.164268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:31.164268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:31.164268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:31.164268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:31.164268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:31.164268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:31.164268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:31.164268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:31.164268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:31.164805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:31.165662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:31.166142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:31.166142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:31.166142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:31.166142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:31.166142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:31.166142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:33.819111 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:33.820045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:33.820045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:33.820876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:33.821071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:33.821298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.821919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:33.821919 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.821919 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:33.821919 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:33.821919 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.821919 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:33.821919 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.821919 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:33.823383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.823383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:33.823383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:33.823383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:33.823383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:33.823383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:33.823383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:33.823383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.823383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:33.823383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:33.823383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:33.823383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.824030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:33.824030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:33.824030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:33.824030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:33.824030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:33.824030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:33.824030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:33.824030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:33.824030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.824030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:33.824030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:33.824030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.824030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:33.825076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:33.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.828089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:33.828089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:33.828089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.828089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:33.828089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.828089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:33.828089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:33.828089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:33.828089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:33.828089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:51:33.829359 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:33.829359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.829359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:33.829359 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.829359 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:33.829359 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:33.829359 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.829359 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:33.829359 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.829359 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:33.829655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:33.830127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:33.830127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:33.830127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.830127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:33.830127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:33.830127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.830127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:33.830127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.830127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:33.830531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.830531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:33.830531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:33.830531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:33.830531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:33.830531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:33.830531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:33.830531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:51:33.830531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:33.830531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:33.830531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.830531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:33.830962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:33.831389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:33.831879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:33.832389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:33.832389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:33.832389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:33.832389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:33.832389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:33.832389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.832389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:33.832389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:33.832389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.832389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:33.832389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.832389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:33.835168 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:33.835599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:33.835736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:33.835907 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:33.836202 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:33.836514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.836898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:33.836898 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.836898 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:33.836898 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:33.836898 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.836898 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:33.836898 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.836898 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:33.837616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:33.837964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:33.837964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:33.837964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.837964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:33.837964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:33.837964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.837964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:33.837964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.837964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:33.838484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.838484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:33.838484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:33.838484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:33.838484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:33.838484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:33.838484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:33.838484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:33.838484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:33.838484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.838484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:33.838740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:33.838977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.838977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:33.838977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.838977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:33.839514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:33.839861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:33.840224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.840224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:33.840224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.840224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:33.841040 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:33.841312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:33.841418 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:33.841562 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:33.841825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:33.842031 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:33.842162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.842610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:33.842610 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.842610 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:33.842610 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:33.842610 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.842610 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:33.842610 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.842610 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:33.843374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.843374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:33.843374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:33.843374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:33.843374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:33.843374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:33.843374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:33.843374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.843374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:33.843374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:33.843374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:33.843374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.843635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:33.843635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:33.843635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:33.843635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:33.843635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:33.843635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:33.843635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:33.843635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:33.843635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.843635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:33.843635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:33.843635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.843635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:33.843865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.843865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:33.844311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:33.844636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:33.845007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:33.845007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:33.845007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:36.127561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:36.128175 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:36.128260 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:36.130028 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:36.168468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:36.168711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:36.169396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:36.169396 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:36.169396 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:36.169396 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:36.169396 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:36.169396 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:36.169396 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:36.169396 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:36.170490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:36.170490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:36.170490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:36.170490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:36.170490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:36.170490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:36.170490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:36.170490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:36.170490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:36.170490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:36.170490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:36.170490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:36.170908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:36.170908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:36.170908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:36.170908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:36.170908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:36.170908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:36.170908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:36.170908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:36.170908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:36.170908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:36.170908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:36.170908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:36.170908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:36.171286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:36.171717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:36.172200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:36.172200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:36.172200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:36.172200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:36.172200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:36.172200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:38.819815 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:38.820691 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:38.820691 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:38.821475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:38.821724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:38.821885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.822546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:38.822546 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.822546 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:38.822546 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:38.822546 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.822546 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:38.822546 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.822546 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:38.824102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.824102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:38.824102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:38.824102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:38.824102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:38.824102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:38.824102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:38.824102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.824102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:38.824102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:38.824102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:38.824102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.824496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:38.824496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:38.824496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:38.824496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:38.824496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:38.824496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:38.824496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:38.824496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:38.824496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.824496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:38.824496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:38.824496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.824496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:38.824832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.824832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:38.826031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:38.826825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.827555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:38.827555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.827555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:38.827555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:38.827555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:38.827555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:38.827555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:51:38.827555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:38.827825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.828379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:38.828379 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.828379 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:38.828379 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:38.828379 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.828379 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:38.828379 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.828379 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:38.829128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:38.829601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:38.829601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:38.829601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.829601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:38.829601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:38.829601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.829601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:38.829601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.829601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:38.830026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.830026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:38.830026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:38.830026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:38.830026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:38.830026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:38.830026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:38.830026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:51:38.830026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:38.830026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:38.830026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.830026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:38.830444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:38.830885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.830885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:38.830885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.830885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:38.831092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:38.831546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:38.832022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.832022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:38.832022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:38.832022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.832022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:38.832022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.832022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:38.835674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:38.835916 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:38.836065 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:38.836248 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:38.837350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:38.837641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.838275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:38.838275 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.838275 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:38.838275 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:38.838275 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.838275 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:38.838275 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.838275 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:38.839019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:38.839622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:38.839622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:38.839622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.839622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:38.839622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:38.839622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.839622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:38.839622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.839622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:38.840394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.840394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:38.840394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:38.840394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:38.840394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:38.840394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:38.840394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:38.840394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:38.840394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:38.840394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.840394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:38.840710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:38.841078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.841078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:38.841078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.841078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:38.841892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:38.842374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:38.842920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.842920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:38.842920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.842920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:38.842920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:38.843286 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:38.843439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:38.843633 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:38.844428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:38.845036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.845912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:38.845912 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.845912 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:38.845912 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:38.845912 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.845912 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:38.845912 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.845912 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:38.846934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.846934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:38.846934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:38.846934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:38.846934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:38.846934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:38.846934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:38.846934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.846934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:38.846934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:38.846934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:38.846934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.847336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:38.847336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:38.847336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:38.847336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:38.847336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:38.847336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:38.847336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:38.847336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:38.847336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.847336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:38.847336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:38.847336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.847336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:38.847692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.847692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:38.847914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:38.848392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:38.848921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:38.848921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:38.848921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:41.128342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:41.128900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:41.129024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:41.129889 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:41.165165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:41.165342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:41.165798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:41.165798 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:41.165798 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:41.165798 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:41.165798 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:41.165798 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:41.165798 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:41.165798 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:41.166473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:41.166888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:41.166888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:41.166888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:41.166888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:41.166888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:41.166888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:41.166888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:41.166888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:41.166888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:41.166888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:41.166888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:41.166888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:41.166888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:41.167299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:41.167762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:41.168344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:41.168344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:41.168344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:41.168344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:41.168344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:41.168344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:43.820495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:43.821175 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:43.821175 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:43.821820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:43.822030 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:43.822181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.822771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:43.822771 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.822771 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:43.822771 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:43.822771 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.822771 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:43.822771 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.822771 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:43.823525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.823525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:43.823525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:43.823525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:43.823525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:43.823525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:43.823525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:43.823525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.823525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:43.823525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:43.823525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:43.823525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:43.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:43.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:43.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:43.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:43.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:43.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:43.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:43.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:43.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:43.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.823939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:43.824393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:43.824900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.825520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:43.825520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:43.825520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.825520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:43.825520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.825520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:43.826687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:43.826952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:43.827038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:43.827124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:51:43.827496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:43.827677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.828199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:43.828199 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.828199 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:43.828199 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:43.828199 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.828199 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:43.828199 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.828199 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:43.828777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:43.829286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:43.829286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:43.829286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.829286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:43.829286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:43.829286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.829286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:43.829286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.829286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:43.829574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.829574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:43.829574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:43.829574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:43.829574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:43.829574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:43.829574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:43.829574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:51:43.829574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:43.829574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:43.829574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.829574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:43.829924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:43.830296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:43.830728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:43.831278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:43.831278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:43.831278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:43.831278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:43.831278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:43.831278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.831278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:43.831278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:43.831278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.831278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:43.831278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.831278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:43.835980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:43.837372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:43.838110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:43.838110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:43.838110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:43.838298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.838703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:43.838703 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.838703 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:43.838703 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:43.838703 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.838703 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:43.838703 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.838703 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:43.839592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:43.840112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:43.840112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:43.840112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.840112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:43.840112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:43.840112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.840112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:43.840112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.840112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:43.840659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.840659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:43.840659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:43.840659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:43.840659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:43.840659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:43.840659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:43.840659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:43.840659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:43.840659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.840659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:43.841157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:43.841577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.841577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:43.841577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.841577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:43.841719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:43.842250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:43.842797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.842797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:43.842797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.842797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:43.843347 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:43.843623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:43.843745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:43.843909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:43.844261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:43.844524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:43.844595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.845082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:43.845082 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.845082 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:43.845082 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:43.845082 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.845082 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:43.845082 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.845082 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:43.846059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.846059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:43.846059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:43.846059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:43.846059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:43.846059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:43.846059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:43.846059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.846059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:43.846059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:43.846059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:43.846059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.846505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:43.846505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:43.846505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:43.846505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:43.846505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:43.846505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:43.846505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:43.846505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:43.846505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.846505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:43.846505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:43.846505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.846505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:43.846850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.846850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:43.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:43.847849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:43.848444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:43.848444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:43.848444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:46.129240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:46.130369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:46.130369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:46.130938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:46.177022 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:46.177291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:46.177961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:46.177961 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:46.177961 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:46.177961 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:46.177961 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:46.177961 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:46.177961 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:46.177961 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:46.178713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:46.178713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:46.178713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:46.178713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:46.178713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:46.178713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:46.178713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:46.178713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:46.178713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:46.178713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:46.178713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:46.178713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:46.179102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:46.179102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:46.179102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:46.179102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:46.179102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:46.179102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:46.179102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:46.179102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:46.179102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:46.179102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:46.179102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:46.179102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:46.179102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:46.179465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:46.179465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:46.179681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:46.180367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:46.181092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:46.181092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:46.181092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:48.821387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:48.822166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:48.822317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:48.823913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:48.824294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:48.824573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.825574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:48.825574 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.825574 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:48.825574 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:48.825574 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.825574 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:48.825574 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.825574 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:48.826909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.826909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:48.826909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:48.826909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:48.826909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:48.826909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:48.826909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:48.826909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.826909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:48.826909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:48.826909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:48.826909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.827353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:48.827353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:48.827353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:48.827353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:48.827353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:48.827353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:48.827353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:48.827353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:48.827353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.827353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:48.827353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:48.827353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.827353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:48.827736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.827736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:48.828528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:48.829646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.830149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:48.830149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.830149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:48.830149 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:48.830149 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:48.830149 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:48.830149 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:51:48.830149 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:48.830360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.830458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:48.830458 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.830458 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:48.830458 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:48.830458 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.830458 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:48.830458 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.830458 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:48.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:48.831497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:48.831497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:48.831497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.831497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:48.831497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:48.831497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.831497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:48.831497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.831497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:48.831919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.831919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:48.831919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:48.831919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:48.831919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:48.831919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:48.831919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:48.831919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:51:48.831919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:48.831919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:48.831919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.831919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:48.832172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:48.832447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.832447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:48.832447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.832447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:48.832901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:48.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:48.833592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.833592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:48.833592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:48.833592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.833592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:48.833592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.833592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:48.837442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:48.837707 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:48.837836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:48.837981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:48.838470 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:48.838824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.839500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:48.839500 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.839500 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:48.839500 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:48.839500 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.839500 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:48.839500 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.839500 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:48.840023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:48.840408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:48.840408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:48.840408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.840408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:48.840408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:48.840408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.840408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:48.840408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.840408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:48.840638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.840638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:48.840638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:48.840638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:48.840638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:48.840638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:48.840638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:48.840638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:48.840638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:48.840638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.840638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:48.840928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:48.841293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.841293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:48.841293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.841293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:48.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:48.841914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:48.842389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.842389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:48.842389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.842389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:48.843772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:48.844087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:48.844200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:48.844379 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:48.844774 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:48.845121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:48.845402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.846066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:48.846066 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.846066 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:48.846066 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:48.846066 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.846066 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:48.846066 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.846066 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:48.846915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.846915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:48.846915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:48.846915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:48.846915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:48.846915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:48.846915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:48.846915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.846915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:48.846915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:48.846915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:48.846915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.847266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:48.847266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:48.847266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:48.847266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:48.847266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:48.847266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:48.847266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:48.847266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:48.847266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.847266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:48.847266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:48.847266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.847266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:48.847623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.847623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:48.847906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:48.848336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:48.848937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:48.848937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:48.848937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:51.130275 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:51.131649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:51.131649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:51.132401 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:51.167529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:51.167949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:51.168850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:51.168850 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:51.168850 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:51.168850 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:51.168850 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:51.168850 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:51.168850 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:51.168850 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:51.169579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:51.169579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:51.169579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:51.169579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:51.169579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:51.169579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:51.169579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:51.169579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:51.169579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:51.169579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:51.169579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:51.169579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:51.170511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:51.170511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:51.170511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:51.170511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:51.170511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:51.170511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:51.170511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:51.170511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:51.170511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:51.170511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:51.170511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:51.170511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:51.170511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:51.171087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:51.171793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:51.172894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:51.172894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:51.172894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:51.172894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:51.172894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:51.172894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:53.822302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:53.822770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:53.822770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:53.823577 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:53.823804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:53.823971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.824650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:53.824650 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.824650 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:53.824650 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:53.824650 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.824650 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:53.824650 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.824650 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:53.825664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.825664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:53.825664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:53.825664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:53.825664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:53.825664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:53.825664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:53.825664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.825664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:53.825664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:53.825664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:53.825664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.825895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:53.825895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:53.825895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:53.825895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:53.825895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:53.825895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:53.825895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:53.825895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:53.825895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.825895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:53.825895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:53.825895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.825895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:53.826146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.826146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:53.826828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:53.827160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.827510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:53.827510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.827510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:53.829006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:53.829261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:53.829385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:53.829509 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:51:53.829862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:53.830116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.830619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:53.830619 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.830619 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:53.830619 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:53.830619 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.830619 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:53.830619 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.830619 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:53.831435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:53.831720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:53.831720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:53.831720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.831720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:53.831720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:53.831720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.831720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:53.831720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.831720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:53.832235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.832235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:53.832235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:53.832235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:53.832235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:53.832235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:53.832235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:53.832235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:51:53.832235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:53.832235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:53.832235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.832235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:53.832563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:53.833110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.833110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:53.833110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.833110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:53.833305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:53.833661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:53.834493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.834493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:53.834493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:53.834493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.834493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:53.834493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.834493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:53.837762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:53.837992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:53.838125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:53.838261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:53.838521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:53.838689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.839065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:53.839065 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.839065 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:53.839065 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:53.839065 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.839065 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:53.839065 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.839065 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:53.839812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:53.840081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:53.840081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:53.840081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.840081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:53.840081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:53.840081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.840081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:53.840081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.840081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:53.840707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.840707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:53.840707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:53.840707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:53.840707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:53.840707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:53.840707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:53.840707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:53.840707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:53.840707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.840707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:53.840924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:53.841179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.841179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:53.841179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.841179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:53.841808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:53.842145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:53.842493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.842493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:53.842493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.842493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:53.844095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:53.844344 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:53.844487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:53.844608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:53.844863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:53.845134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:53.845471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.846152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:53.846152 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.846152 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:53.846152 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:53.846152 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.846152 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:53.846152 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.846152 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:53.846987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.846987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:53.846987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:53.846987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:53.846987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:53.846987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:53.846987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:53.846987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.846987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:53.846987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:53.846987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:53.846987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.847201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:53.847201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:53.847201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:53.847201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:53.847201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:53.847201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:53.847201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:53.847201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:53.847201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.847201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:53.847201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:53.847201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.847201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:53.847439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.847439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:53.847977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:53.848281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:53.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:53.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:53.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:56.130905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:56.131493 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:56.131493 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:56.132399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:56.166159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:56.166659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:56.167541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:56.167541 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:56.167541 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:56.167541 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:56.167541 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:56.167541 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:56.167541 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:56.167541 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:56.168325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:56.168325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:56.168325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:56.168325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:56.168325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:56.168325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:56.168325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:56.168325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:56.168325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:56.168325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:56.168325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:56.168325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:56.169391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:56.169391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:56.169391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:56.169391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:56.169391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:56.169391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:56.169391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:56.169391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:56.169391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:56.169391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:56.169391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:56.169391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:56.169391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:56.170423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:56.171243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:56.172054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:56.172054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:56.172054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:56.172054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:56.172054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:56.172054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:58.822788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:58.823539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:58.823736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:58.824687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:58.824890 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:58.825048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.825790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:58.825790 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.825790 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:58.825790 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:58.825790 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.825790 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:58.825790 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.825790 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:58.827304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.827304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:58.827304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:58.827304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:58.827304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:58.827304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:58.827304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:58.827304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.827304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:58.827304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:58.827304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:58.827304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.828007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:58.828007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:58.828007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:58.828007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:58.828007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:58.828007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:58.828007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:58.828007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:58.828007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.828007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:58.828007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:58.828007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.828007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:58.828501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:58.829147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.829889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:58.829889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:58.829889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.829889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:58.829889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.829889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:58.830755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:58.830997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:58.831110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:58.831300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:51:58.831625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:58.831809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.832351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:58.832351 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.832351 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:58.832351 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:58.832351 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.832351 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:58.832351 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.832351 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:58.833129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:58.833844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:58.833844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:58.833844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.833844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:58.833844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:58.833844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.833844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:58.833844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.833844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:58.834261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.834261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:58.834261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:58.834261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:58.834261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:58.834261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:58.834261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:58.834261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:51:58.834261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:58.834261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:58.834261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.834261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:58.834833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:58.835396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:58.835939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:58.836806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:58.836806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:58.836806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:58.836806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:58.836806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:58.836806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.836806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:58.836806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:58.836806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.836806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:58.836806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.836806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:58.838232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:58.838432 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:58.838555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:58.838695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:58.839024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:58.839274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.839743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:58.839743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.839743 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:58.839743 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:58.839743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.839743 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:58.839743 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.839743 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:51:58.840624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:51:58.841230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:58.841230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:51:58.841230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.841230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:51:58.841230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:51:58.841230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.841230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:51:58.841230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.841230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:51:58.841625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.841625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:58.841625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:58.841625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:58.841625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:58.841625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:58.841625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:58.841625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:58.841625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:58.841625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.841625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:58.842089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:58.842662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.842662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:58.842662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.842662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:58.842840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:58.843398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:58.843975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.843975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:58.843975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.843975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:51:58.844442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:51:58.844676 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:51:58.844786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:51:58.844951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:51:58.845272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:58.845494 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:51:58.845637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:51:58.846098 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.846098 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:51:58.846098 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:51:58.846098 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.846098 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:51:58.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:51:58.846998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.846998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:51:58.846998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:58.846998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:51:58.846998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:51:58.846998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:58.846998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:58.846998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.846998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:58.846998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:58.846998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:51:58.846998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:51:58.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:58.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:51:58.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:58.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:51:58.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:51:58.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:58.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:51:58.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:51:58.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:51:58.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.847393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:51:58.847691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.847691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:51:58.848343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:51:58.848804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:51:58.849239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:51:58.849239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:51:58.849239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:01.132119 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:01.133065 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:01.133065 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:01.133748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:01.164249 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:01.164656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:01.165702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:01.165702 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:01.165702 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:01.165702 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:01.165702 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:01.165702 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:01.165702 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:01.165702 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:01.166586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:01.166586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:01.166586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:01.166586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:01.166586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:01.166586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:01.166586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:01.166586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:01.166586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:01.166586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:01.166586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:01.166586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:01.167258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:01.167258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:01.167258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:01.167258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:01.167258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:01.167258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:01.167258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:01.167258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:01.167258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:01.167258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:01.167258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:01.167258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:01.167258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:01.167849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:01.168545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:01.169353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:01.169353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:01.169353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:01.169353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:01.169353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:01.169353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:03.823440 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:03.824434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:03.824434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:03.824750 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:03.825053 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:03.825321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.826121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:03.826121 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.826121 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:03.826121 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:03.826121 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.826121 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:03.826121 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.826121 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:03.827166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.827166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:03.827166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:03.827166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:03.827166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:03.827166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:03.827166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:03.827166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.827166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:03.827166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:03.827166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:03.827166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.827550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:03.827550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:03.827550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:03.827550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:03.827550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:03.827550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:03.827550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:03.827550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:03.827550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.827550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:03.827550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:03.827550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.827550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:03.827843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.827843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:03.828250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:03.828585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.828930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:03.828930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.828930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:03.830973 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:03.831155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:03.831285 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:03.831368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:52:03.831851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:03.832142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.832825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:03.832825 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.832825 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:03.832825 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:03.832825 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.832825 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:03.832825 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.832825 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:03.833398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:03.833730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:03.833730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:03.833730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.833730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:03.833730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:03.833730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.833730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:03.833730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.833730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:03.834026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.834026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:03.834026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:03.834026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:03.834026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:03.834026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:03.834026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:03.834026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:52:03.834026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:03.834026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:03.834026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.834026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:03.834272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:03.834559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.834559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:03.834559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.834559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:03.834850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:03.835289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:03.836048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.836048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:03.836048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:03.836048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.836048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:03.836048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.836048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:03.838465 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:03.838814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:03.838926 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:03.838993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:03.839376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:03.839597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.840122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:03.840122 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.840122 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:03.840122 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:03.840122 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.840122 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:03.840122 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.840122 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:03.841045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:03.841572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:03.841572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:03.841572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.841572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:03.841572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:03.841572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.841572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:03.841572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.841572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:03.842059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.842059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:03.842059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:03.842059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:03.842059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:03.842059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:03.842059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:03.842059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:03.842059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:03.842059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.842059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:03.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:03.842868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.842868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:03.842868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.842868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:03.843396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:03.843950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:03.844629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.844629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:03.844629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.844629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:03.844870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:03.845120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:03.845307 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:03.845522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:03.846031 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:03.846459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:03.846740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.847540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:03.847540 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.847540 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:03.847540 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:03.847540 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.847540 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:03.847540 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.847540 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:03.848442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.848442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:03.848442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:03.848442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:03.848442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:03.848442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:03.848442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:03.848442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.848442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:03.848442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:03.848442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:03.848442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.849005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:03.849005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:03.849005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:03.849005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:03.849005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:03.849005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:03.849005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:03.849005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:03.849005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.849005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:03.849005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:03.849005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.849005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:03.849586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:03.850243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:03.851045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:03.851045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:03.851045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:03.851045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:03.851045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:03.851045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:06.132405 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:06.133304 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:06.133304 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:06.133548 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:06.134763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:52:06.160276 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:06.198686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:06.199790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:06.200467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:06.200467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:06.200467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:06.200467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:06.200467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:06.200467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:06.200467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:06.200467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:06.200467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:06.200467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:06.200467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:06.200467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:06.200467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:06.201064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:06.201975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:06.201975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:06.201975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:06.201975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:06.201975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:06.201975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:06.201975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:06.201975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:06.201975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:06.201975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:06.201975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:06.201975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:08.823981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:08.824532 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:08.825324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:08.825730 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:08.826112 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:08.826414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.827230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:08.827230 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.827230 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:08.827230 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:08.827230 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.827230 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:08.827230 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.827230 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:08.828294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.828294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:08.828294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:08.828294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:08.828294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:08.828294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:08.828294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:08.828294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.828294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:08.828294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:08.828294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:08.828294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.828844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:08.828844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:08.828844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:08.828844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:08.828844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:08.828844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:08.828844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:08.828844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:08.828844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.828844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:08.828844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:08.828844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.828844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:08.829433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:08.830071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.831030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:08.831030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:08.831030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.831030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:08.831030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.831030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:08.831443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:08.831741 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:08.831860 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:08.831973 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:52:08.832450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:08.832725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.833368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:08.833368 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.833368 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:08.833368 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:08.833368 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.833368 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:08.833368 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.833368 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:08.834125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:08.834641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:08.834641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:08.834641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.834641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:08.834641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:08.834641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.834641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:08.834641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.834641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:08.835022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.835022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:08.835022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:08.835022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:08.835022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:08.835022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:08.835022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:08.835022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:52:08.835022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:08.835022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:08.835022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.835022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:08.835380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:08.835719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.835719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:08.835719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.835719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:08.836073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:08.836501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:08.836990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.836990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:08.836990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:08.836990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.836990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:08.836990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.836990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:08.838972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:08.839280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:08.839408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:08.839540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:08.839939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:08.840345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.841140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:08.841140 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.841140 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:08.841140 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:08.841140 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.841140 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:08.841140 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.841140 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:08.841905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:08.842405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:08.842405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:08.842405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.842405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:08.842405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:08.842405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.842405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:08.842405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.842405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:08.842860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.842860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:08.842860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:08.842860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:08.842860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:08.842860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:08.842860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:08.842860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:08.842860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:08.842860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.842860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:08.843201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:08.843581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.843581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:08.843581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.843581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:08.843959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:08.844561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:08.845021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.845021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:08.845021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.845021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:08.845347 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:08.845632 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:08.845745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:08.846013 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:08.846442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:08.846741 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:08.846907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.847446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:08.847446 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.847446 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:08.847446 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:08.847446 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.847446 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:08.847446 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.847446 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:08.848285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.848285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:08.848285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:08.848285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:08.848285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:08.848285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:08.848285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:08.848285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.848285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:08.848285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:08.848285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:08.848285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.848636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:08.848636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:08.848636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:08.848636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:08.848636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:08.848636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:08.848636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:08.848636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:08.848636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.848636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:08.848636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:08.848636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.848636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:08.848971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.848971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:08.849277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:08.849703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:08.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:08.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:08.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:11.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:11.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:11.133972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:11.134537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:11.173426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:11.173732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:11.174665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:11.174665 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:11.174665 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:11.174665 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:11.174665 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:11.174665 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:11.174665 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:11.174665 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:11.175896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:11.175896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:11.175896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:11.175896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:11.175896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:11.175896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:11.175896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:11.175896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:11.175896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:11.175896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:11.175896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:11.175896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:11.176205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:11.176205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:11.176205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:11.176205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:11.176205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:11.176205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:11.176205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:11.176205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:11.176205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:11.176205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:11.176205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:11.176205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:11.176205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:11.176480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:11.176480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:11.177025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:11.177374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:11.177715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:11.177715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:11.177715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:13.824570 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:13.825133 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:13.825133 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:13.825855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:13.826019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:13.826139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.827557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:13.827557 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.827557 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:13.827557 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:13.827557 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.827557 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:13.827557 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.827557 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:13.828611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.828611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:13.828611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:13.828611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:13.828611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:13.828611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:13.828611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:13.828611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.828611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:13.828611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:13.828611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:13.828611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.828960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:13.828960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:13.828960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:13.828960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:13.828960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:13.828960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:13.828960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:13.828960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:13.828960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.828960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:13.828960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:13.828960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.828960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:13.829274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.829274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:13.829711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:13.830017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.830385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:13.830385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.830385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:13.831818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:13.832082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:13.832196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:13.832343 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:52:13.832772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:13.832946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.833409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:13.833409 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.833409 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:13.833409 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:13.833409 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.833409 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:13.833409 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.833409 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:13.834185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:13.834664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:13.834664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:13.834664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.834664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:13.834664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:13.834664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.834664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:13.834664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.834664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:13.835094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.835094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:13.835094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:13.835094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:13.835094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:13.835094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:13.835094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:13.835094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:52:13.835094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:13.835094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:13.835094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.835094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:13.835512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:13.835915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.835915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:13.835915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.835915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:13.836168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:13.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:13.837135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.837135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:13.837135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:13.837135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.837135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:13.837135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.837135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:13.839331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:13.839579 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:13.839695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:13.839829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:13.840144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:13.840365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.840874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:13.840874 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.840874 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:13.840874 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:13.840874 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.840874 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:13.840874 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.840874 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:13.841658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:13.842102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:13.842102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:13.842102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.842102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:13.842102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:13.842102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.842102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:13.842102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.842102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:13.842606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.842606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:13.842606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:13.842606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:13.842606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:13.842606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:13.842606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:13.842606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:13.842606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:13.842606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.842606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:13.842922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:13.843341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.843341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:13.843341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.843341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:13.843727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:13.844223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:13.845066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.845066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:13.845066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.845066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:13.846122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:13.846384 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:13.846521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:13.846670 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:13.846936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:13.847349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:13.847499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.847967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:13.847967 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.847967 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:13.847967 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:13.847967 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.847967 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:13.847967 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.847967 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:13.848829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.848829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:13.848829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:13.848829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:13.848829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:13.848829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:13.848829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:13.848829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.848829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:13.848829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:13.848829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:13.848829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.849232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:13.849232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:13.849232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:13.849232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:13.849232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:13.849232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:13.849232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:13.849232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:13.849232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.849232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:13.849232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:13.849232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.849232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:13.849556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.849556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:13.849824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:13.850275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:13.850970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:13.850970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:13.850970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:16.133438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:16.134017 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:16.134017 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:16.134420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:16.162432 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:16.162619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:16.163117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:16.163117 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:16.163117 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:16.163117 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:16.163117 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:16.163117 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:16.163117 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:16.163117 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:16.163860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:16.163860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:16.163860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:16.163860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:16.163860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:16.163860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:16.163860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:16.163860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:16.163860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:16.163860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:16.163860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:16.163860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:16.164315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:16.164315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:16.164315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:16.164315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:16.164315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:16.164315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:16.164315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:16.164315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:16.164315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:16.164315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:16.164315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:16.164315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:16.164315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:16.164710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:16.165492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:16.166049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:16.166049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:16.166049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:16.166049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:16.166049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:16.166049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:18.825073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:18.825975 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:18.826140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:18.827121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:18.827390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:18.827537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.828364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:18.828364 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.828364 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:18.828364 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:18.828364 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.828364 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:18.828364 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.828364 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:18.830291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.830291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:18.830291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:18.830291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:18.830291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:18.830291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:18.830291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:18.830291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.830291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:18.830291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:18.830291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:18.830291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.830777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:18.830777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:18.830777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:18.830777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:18.830777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:18.830777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:18.830777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:18.830777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:18.830777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.830777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:18.830777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:18.830777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.830777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:18.831113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.831113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:18.835540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:18.836078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:18.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:18.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:18.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:18.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:18.836651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:52:18.837022 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:18.837220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.837871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:18.837871 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.837871 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:18.837871 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:18.837871 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.837871 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:18.837871 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.837871 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:18.838700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:18.839265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:18.839265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:18.839265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.839265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:18.839265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:18.839265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.839265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:18.839265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.839265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:52:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:18.839963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:18.840391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.840391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:18.840391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.840391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:18.840946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:18.841803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:18.842709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.842709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:18.842709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:18.842709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.842709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:18.842709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.842709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:18.842709 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:18.842709 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:18.842709 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:18.843164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:18.843164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:18.843164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.843164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:18.843164 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.843164 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:18.843164 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:18.843164 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.843164 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:18.843164 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.843164 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:18.843164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:18.843705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.844860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:18.844860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:18.844860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.844860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:18.844860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.844860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:18.844860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.844860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:18.844860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:18.844860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:18.844860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:18.844860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:18.844860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:18.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:18.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:18.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:18.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:18.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:18.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:18.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:18.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:18.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:18.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:18.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:18.847096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:18.847096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:18.847096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.847096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:18.847096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:18.847096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.847096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:18.847096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.847096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:18.847096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.847096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:18.847096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.847572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.848225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:18.848782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:18.848782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:18.848782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:18.848782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:18.848782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:18.848782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:18.849003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:18.849554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:18.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:18.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:18.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:18.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:18.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:18.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:18.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:18.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:18.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:18.850677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:18.851197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:18.851197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:18.851197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:18.851197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:18.851197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:18.851197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:18.851197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:18.851197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:18.851197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:18.851197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:18.851197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:18.851197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:21.134003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:21.134909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:21.134909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:21.136412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:21.170618 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:21.170902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:21.171514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:21.171514 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:21.171514 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:21.171514 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:21.171514 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:21.171514 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:21.171514 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:21.171514 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:21.172272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:21.172272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:21.172272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:21.172272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:21.172272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:21.172272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:21.172272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:21.172272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:21.172272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:21.172272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:21.172272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:21.172272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:21.172717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:21.172717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:21.172717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:21.172717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:21.172717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:21.172717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:21.172717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:21.172717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:21.172717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:21.172717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:21.172717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:21.172717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:21.172717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:21.173141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:21.173692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:21.174285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:21.174285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:21.174285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:21.174285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:21.174285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:21.174285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:23.825860 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:23.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:23.826842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:23.827872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:23.828154 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:23.828494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.829288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:23.829288 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.829288 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:23.829288 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:23.829288 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.829288 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:23.829288 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.829288 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:23.830860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.830860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:23.830860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:23.830860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:23.830860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:23.830860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:23.830860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:23.830860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.830860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:23.830860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:23.830860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:23.830860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.831165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:23.831165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:23.831165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:23.831165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:23.831165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:23.831165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:23.831165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:23.831165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:23.831165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.831165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:23.831165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:23.831165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.831165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:23.831427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.831427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:23.831976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:23.832325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.832703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:23.832703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.832703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:23.837340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:23.837340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:23.837340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:23.837340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:52:23.837340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:23.837588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.838554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:23.838554 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.838554 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:23.838554 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:23.838554 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.838554 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:23.838554 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.838554 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:23.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:23.839157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:23.839157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:23.839157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.839157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:23.839157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:23.839157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.839157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:23.839157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.839157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:23.839731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.839731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:23.839731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:23.839731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:23.839731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:23.839731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:23.839731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:23.839731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:52:23.839731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:23.839731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:23.839731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.839731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:23.840006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:23.840287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.840287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:23.840287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.840287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:23.840840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:23.841167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:23.841555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.841555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:23.841555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:23.841555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.841555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:23.841555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.841555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:23.841555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:23.841555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:23.841753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:23.841753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:23.842010 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:23.842261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.842761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:23.842761 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.842761 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:23.842761 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:23.842761 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.842761 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:23.842761 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.842761 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:23.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:23.843811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:23.843811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:23.843811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.843811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:23.843811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:23.843811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.843811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:23.843811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.843811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:23.844466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.844466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:23.844466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:23.844466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:23.844466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:23.844466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:23.844466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:23.844466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:23.844466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:23.844466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.844466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:23.844724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:23.845010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.845010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:23.845010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.845010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:23.845603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:23.845966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:23.846331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.846331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:23.846331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.846331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:23.847308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:23.847308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:23.847308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:23.847308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:23.847882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:23.847882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:23.847882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.848321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:23.848321 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.848321 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:23.848321 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:23.848321 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.848321 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:23.848321 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.848321 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:23.849255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.849255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:23.849255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:23.849255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:23.849255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:23.849255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:23.849255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:23.849255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.849255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:23.849255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:23.849255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:23.849255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.849527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:23.849527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:23.849527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:23.849527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:23.849527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:23.849527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:23.849527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:23.849527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:23.849527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.849527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:23.849527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:23.849527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.849527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:23.849755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.849755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:23.850307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:23.850647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:23.851018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:23.851018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:23.851018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:26.134636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:26.135596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:26.135596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:26.136004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:26.172437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:26.172606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:26.173074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:26.173074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:26.173074 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:26.173074 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:26.173074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:26.173074 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:26.173074 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:26.173074 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:26.173902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:26.173902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:26.173902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:26.173902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:26.173902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:26.173902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:26.173902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:26.173902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:26.173902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:26.173902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:26.173902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:26.173902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:26.174528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:26.174528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:26.174528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:26.174528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:26.174528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:26.174528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:26.174528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:26.174528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:26.174528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:26.174528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:26.174528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:26.174528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:26.174528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:26.175066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:26.175747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:26.176565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:26.176565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:26.176565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:26.176565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:26.176565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:26.176565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:28.826468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:28.827412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:28.827412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:28.828327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:28.828669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:28.828922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.829877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:28.829877 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.829877 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:28.829877 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:28.829877 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.829877 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:28.829877 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.829877 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:28.831943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.831943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:28.831943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:28.831943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:28.831943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:28.831943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:28.831943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:28.831943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.831943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:28.831943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:28.831943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:28.831943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.832698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:28.832698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:28.832698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:28.832698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:28.832698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:28.832698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:28.832698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:28.832698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:28.832698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.832698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:28.832698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:28.832698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.832698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:28.833343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.833343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:28.833678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:28.834791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.835853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:28.835853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.835853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:28.836811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:28.837148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:28.837324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:28.837504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:52:28.837924 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:28.838189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.838895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:28.838895 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.838895 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:28.838895 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:28.838895 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.838895 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:28.838895 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.838895 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:28.839802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:28.840756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:28.840756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:28.840756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.840756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:28.840756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:28.840756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.840756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:28.840756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.840756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:28.841356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.841356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:28.841356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:28.841356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:28.841356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:28.841356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:28.841356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:28.841356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:52:28.841356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:28.841356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:28.841356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.841356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:28.842060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:28.842833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:28.843610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:28.844627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:28.845140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:28.845140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:28.845140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:28.845140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.845140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:28.845140 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.845140 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:28.845140 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:28.845140 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.845140 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:28.845140 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.845140 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:28.846032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:28.846575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:28.846575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:28.846575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.846575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:28.846575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:28.846575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.846575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:28.846575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.846575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:28.847106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.847106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:28.847106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:28.847106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:28.847106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:28.847106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:28.847106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:28.847106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:28.847106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:28.847106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.847106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:28.847578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:28.848061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.848061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:28.848061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.848061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:28.848667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:28.849055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:28.849485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.849485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:28.849485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.849485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:28.849485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:28.849485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:28.849485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:28.849485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:28.849829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:28.850162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:28.850435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.850989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:28.850989 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.850989 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:28.850989 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:28.850989 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.850989 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:28.850989 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.850989 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:28.851666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.851666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:28.851666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:28.851666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:28.851666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:28.851666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:28.851666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:28.851666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.851666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:28.851666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:28.851666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:28.851666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.851978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:28.851978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:28.851978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:28.851978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:28.851978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:28.851978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:28.851978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:28.851978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:28.851978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.851978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:28.851978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:28.851978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.851978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:28.852300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.852300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:28.852475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:28.852848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:28.853291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:28.853291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:28.853291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:31.135255 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:31.135814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:31.136014 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:31.137402 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:31.168338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:31.168754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:31.169650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:31.169650 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:31.169650 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:31.169650 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:31.169650 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:31.169650 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:31.169650 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:31.169650 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:31.170696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:31.170696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:31.170696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:31.170696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:31.170696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:31.170696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:31.170696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:31.170696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:31.170696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:31.170696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:31.170696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:31.170696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:31.171407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:31.171407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:31.171407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:31.171407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:31.171407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:31.171407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:31.171407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:31.171407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:31.171407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:31.171407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:31.171407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:31.171407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:31.171407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:31.172011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:31.172839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:31.173763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:31.173763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:31.173763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:31.173763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:31.173763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:31.173763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:33.826976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:33.827841 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:33.827984 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:33.828749 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:33.828991 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:33.829158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.829956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:33.829956 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.829956 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:33.829956 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:33.829956 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.829956 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:33.829956 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.829956 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:33.831507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.831507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:33.831507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:33.831507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:33.831507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:33.831507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:33.831507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:33.831507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.831507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:33.831507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:33.831507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:33.831507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.831796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:33.831796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:33.831796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:33.831796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:33.831796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:33.831796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:33.831796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:33.831796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:33.831796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.831796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:33.831796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:33.831796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.831796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:33.832045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.832045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:33.833556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:33.834307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.834655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:33.834655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.834655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:33.837354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:33.837591 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:33.837712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:33.837834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:52:33.838282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:33.838501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.839123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:33.839123 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.839123 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:33.839123 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:33.839123 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.839123 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:33.839123 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.839123 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:33.840534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:33.840863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:33.840863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:33.840863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.840863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:33.840863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:33.840863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.840863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:33.840863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.840863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:33.841513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.841513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:33.841513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:33.841513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:33.841513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:33.841513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:33.841513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:33.841513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:52:33.841513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:33.841513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:33.841513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.841513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:33.841768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:33.842007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.842007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:33.842007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.842007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:33.842561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:33.842938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:33.843303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.843303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:33.843303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:33.843303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.843303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:33.843303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.843303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:33.843538 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:33.843758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:33.843872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:33.844026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:33.844406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:33.844674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.845266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:33.845266 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.845266 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:33.845266 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:33.845266 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.845266 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:33.845266 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.845266 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:33.846017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:33.846312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:33.846312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:33.846312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.846312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:33.846312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:33.846312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.846312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:33.846312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.846312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:33.847588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.847588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:33.847588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:33.847588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:33.847588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:33.847588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:33.847588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:33.847588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:33.847588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:33.847588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.847588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:33.847833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:33.848265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.848265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:33.848265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.848265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:33.848685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:33.849021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:33.849641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.849641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:33.849641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.849641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:33.849641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:33.849641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:33.849781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:33.849869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:33.850166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:33.850429 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:33.850582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.851544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:33.851544 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.851544 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:33.851544 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:33.851544 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.851544 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:33.851544 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.851544 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:33.852387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.852387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:33.852387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:33.852387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:33.852387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:33.852387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:33.852387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:33.852387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.852387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:33.852387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:33.852387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:33.852387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.852657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:33.852657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:33.852657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:33.852657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:33.852657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:33.852657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:33.852657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:33.852657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:33.852657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.852657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:33.852657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:33.852657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.852657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:33.852888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.852888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:33.853345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:33.853660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:33.853987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:33.853987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:33.853987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:36.136391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:36.137158 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:36.137158 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:36.139352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:36.172267 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:36.172795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:36.173906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:36.173906 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:36.173906 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:36.173906 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:36.173906 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:36.173906 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:36.173906 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:36.173906 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:36.175648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:36.175648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:36.175648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:36.175648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:36.175648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:36.175648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:36.175648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:36.175648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:36.175648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:36.175648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:36.175648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:36.175648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:36.176453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:36.176453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:36.176453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:36.176453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:36.176453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:36.176453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:36.176453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:36.176453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:36.176453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:36.176453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:36.176453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:36.176453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:36.176453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:36.176962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:36.176962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:36.177350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:36.177886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:36.178630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:36.178630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:36.178630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:38.827912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:38.828768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:38.828768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:38.829944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:38.830238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:38.830658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.831721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:38.831721 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.831721 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:38.831721 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:38.831721 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.831721 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:38.831721 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.831721 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:38.832957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.832957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:38.832957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:38.832957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:38.832957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:38.832957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:38.832957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:38.832957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.832957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:38.832957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:38.832957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:38.832957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.833785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:38.833785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:38.833785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:38.833785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:38.833785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:38.833785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:38.833785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:38.833785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:38.833785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.833785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:38.833785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:38.833785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.833785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:38.834609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:38.835928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.837242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:38.837242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:38.837242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.837242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:38.837242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.837242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:38.838911 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:38.838911 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:38.838911 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:38.839194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:52:38.839684 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:38.839981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.840706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:38.840706 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.840706 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:38.840706 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:38.840706 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.840706 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:38.840706 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.840706 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:38.841900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:38.842477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:38.842477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:38.842477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.842477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:38.842477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:38.842477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.842477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:38.842477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.842477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:38.843327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.843327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:38.843327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:38.843327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:38.843327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:38.843327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:38.843327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:38.843327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:52:38.843327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:38.843327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:38.843327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.843327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:38.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:38.844633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.844633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:38.844633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.844633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:38.844802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:38.845615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:38.846447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.846447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:38.846447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:38.846447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.846447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:38.846447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.846447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:38.846447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:38.846447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:38.846447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:38.846884 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:38.846884 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:38.846884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.846884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:38.846884 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.846884 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:38.846884 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:38.846884 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.846884 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:38.846884 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.846884 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:38.847557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:38.848064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:38.848064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:38.848064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.848064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:38.848064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:38.848064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.848064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:38.848064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.848064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:38.848631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.848631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:38.848631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:38.848631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:38.848631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:38.848631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:38.848631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:38.848631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:38.848631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:38.848631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.848631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:38.849050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:38.849570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.849570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:38.849570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.849570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:38.850070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:38.850770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:38.851502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.851502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:38.851502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.851502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:38.851502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:38.851502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:38.851502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:38.851502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:38.851772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:38.851772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:38.851772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.852165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:38.852165 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.852165 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:38.852165 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:38.852165 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.852165 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:38.852165 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.852165 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:38.853136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.853136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:38.853136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:38.853136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:38.853136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:38.853136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:38.853136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:38.853136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.853136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:38.853136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:38.853136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:38.853136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.853404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:38.853404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:38.853404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:38.853404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:38.853404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:38.853404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:38.853404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:38.853404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:38.853404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.853404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:38.853404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:38.853404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.853404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:38.853629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.853629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:38.854140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:38.854479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:38.854886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:38.854886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:38.854886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:41.136513 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:41.137121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:41.137301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:41.138039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:41.174112 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:41.174349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:41.174946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:41.174946 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:41.174946 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:41.174946 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:41.174946 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:41.174946 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:41.174946 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:41.174946 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:41.175904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:41.175904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:41.175904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:41.175904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:41.175904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:41.175904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:41.175904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:41.175904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:41.175904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:41.175904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:41.175904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:41.175904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:41.176301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:41.176301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:41.176301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:41.176301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:41.176301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:41.176301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:41.176301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:41.176301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:41.176301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:41.176301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:41.176301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:41.176301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:41.176301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:41.176664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:41.176664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:41.177032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:41.177519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:41.178046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:41.178046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:41.178046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:43.828415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:43.829416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:43.829416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:43.829860 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:43.830052 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:43.830170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.830808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:43.830808 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.830808 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:43.830808 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:43.830808 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.830808 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:43.830808 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.830808 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:43.831860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.831860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:43.831860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:43.831860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:43.831860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:43.831860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:43.831860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:43.831860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.831860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:43.831860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:43.831860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:43.831860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.832546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:43.832546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:43.832546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:43.832546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:43.832546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:43.832546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:43.832546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:43.832546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:43.832546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.832546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:43.832546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:43.832546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.832546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:43.833225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:43.834022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.834963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:43.834963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:43.834963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.834963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:43.834963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.834963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:43.838666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:43.838666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:43.838821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:43.838912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:52:43.839384 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:43.839617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.840300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:43.840300 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.840300 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:43.840300 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:43.840300 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.840300 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:43.840300 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.840300 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:43.841069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:43.841617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:43.841617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:43.841617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.841617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:43.841617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:43.841617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.841617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:43.841617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.841617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:43.841955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.841955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:43.841955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:43.841955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:43.841955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:43.841955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:43.841955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:43.841955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:52:43.841955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:43.841955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:43.841955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.841955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:43.842369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:43.842771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.842771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:43.842771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.842771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:43.843000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:43.843504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:43.844027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.844027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:43.844027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:43.844027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.844027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:43.844027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.844027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:43.845296 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:43.845540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:43.845652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:43.845828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:43.846184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:43.846527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.847074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:43.847074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.847074 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:43.847074 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:43.847074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.847074 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:43.847074 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.847074 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:43.847911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:43.848369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:43.848369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:43.848369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.848369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:43.848369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:43.848369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.848369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:43.848369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.848369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:43.849089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.849089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:43.849089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:43.849089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:43.849089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:43.849089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:43.849089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:43.849089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:43.849089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:43.849089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.849089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:43.849573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:43.849987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.849987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:43.849987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.849987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:43.850779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:43.851310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:43.851856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.851856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:43.851856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.851856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:43.851856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:43.851856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:43.851856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:43.851856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:43.852121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:43.852358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:43.852503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.853297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:43.853297 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.853297 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:43.853297 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:43.853297 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.853297 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:43.853297 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.853297 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:43.854588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.854588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:43.854588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:43.854588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:43.854588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:43.854588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:43.854588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:43.854588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.854588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:43.854588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:43.854588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:43.854588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.854966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:43.854966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:43.854966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:43.854966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:43.854966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:43.854966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:43.854966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:43.854966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:43.854966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.854966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:43.854966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:43.854966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.854966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:43.855386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.855386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:43.855833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:43.856330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:43.856887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:43.856887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:43.856887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:46.137250 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:46.138671 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:46.138856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:46.140112 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:46.171157 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:46.171389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:46.171900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:46.171900 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:46.171900 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:46.171900 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:46.171900 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:46.171900 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:46.171900 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:46.171900 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:46.172726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:46.172726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:46.172726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:46.172726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:46.172726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:46.172726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:46.172726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:46.172726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:46.172726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:46.172726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:46.172726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:46.172726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:46.173408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:46.173408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:46.173408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:46.173408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:46.173408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:46.173408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:46.173408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:46.173408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:46.173408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:46.173408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:46.173408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:46.173408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:46.173408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:46.174052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:46.175633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:46.177967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:46.177967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:46.177967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:46.177967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:46.177967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:46.177967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:48.828898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:48.829662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:48.829662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:48.830130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:48.830360 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:48.830484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.831500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:48.831500 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.831500 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:48.831500 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:48.831500 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.831500 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:48.831500 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.831500 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:48.832609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.832609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:48.832609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:48.832609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:48.832609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:48.832609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:48.832609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:48.832609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.832609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:48.832609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:48.832609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:48.832609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.833317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:48.833317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:48.833317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:48.833317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:48.833317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:48.833317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:48.833317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:48.833317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:48.833317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.833317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:48.833317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:48.833317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.833317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:48.833792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.833792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:48.833888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:48.834437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.835061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:48.835061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.835061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:48.838736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:48.839035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:48.839159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:48.839337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:52:48.839676 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:48.839905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.840542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:48.840542 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.840542 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:48.840542 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:48.840542 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.840542 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:48.840542 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.840542 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:48.841410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:48.841853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:48.841853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:48.841853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.841853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:48.841853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:48.841853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.841853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:48.841853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.841853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:48.842294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.842294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:48.842294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:48.842294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:48.842294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:48.842294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:48.842294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:48.842294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:52:48.842294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:48.842294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:48.842294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.842294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:48.842651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:48.843035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.843035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:48.843035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.843035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:48.843406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:48.843856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:48.844379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.844379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:48.844379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:48.844379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.844379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:48.844379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.844379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:48.845630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:48.845879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:48.845999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:48.846133 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:48.846462 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:48.846724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.847275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:48.847275 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.847275 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:48.847275 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:48.847275 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.847275 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:48.847275 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.847275 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:48.848024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:48.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:48.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:48.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:48.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:48.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:48.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:48.848995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.848995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:48.848995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:48.848995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:48.848995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:48.848995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:48.848995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:48.848995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:48.848995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:48.848995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.848995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:48.849344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:48.849733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.849733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:48.849733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.849733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:48.850173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:48.850657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:48.851259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.851259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:48.851259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.851259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:48.851430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:48.851666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:48.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:48.851857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:48.852164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:48.852464 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:48.852637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.853142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:48.853142 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.853142 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:48.853142 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:48.853142 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.853142 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:48.853142 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.853142 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:48.853752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.853752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:48.853752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:48.853752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:48.853752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:48.853752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:48.853752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:48.853752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.853752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:48.853752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:48.853752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:48.853752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.854188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:48.854188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:48.854188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:48.854188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:48.854188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:48.854188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:48.854188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:48.854188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:48.854188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.854188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:48.854188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:48.854188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.854188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:48.854631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:48.855168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:48.855780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:48.855780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:48.855780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:48.855780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:48.855780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:48.855780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:51.138514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:51.139084 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:51.139223 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:51.140065 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:51.179885 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:51.180231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:51.181074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:51.181074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:51.181074 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:51.181074 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:51.181074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:51.181074 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:51.181074 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:51.181074 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:51.182263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:51.182263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:51.182263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:51.182263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:51.182263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:51.182263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:51.182263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:51.182263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:51.182263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:51.182263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:51.182263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:51.182263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:51.182565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:51.182565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:51.182565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:51.182565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:51.182565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:51.182565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:51.182565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:51.182565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:51.182565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:51.182565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:51.182565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:51.182565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:51.182565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:51.182840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:51.182840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:51.183447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:51.183776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:51.184170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:51.184170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:51.184170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:53.829463 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:53.830492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:53.830492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:53.832678 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:53.832897 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:53.833030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.833638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:53.833638 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.833638 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:53.833638 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:53.833638 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.833638 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:53.833638 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.833638 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:53.834709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.834709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:53.834709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:53.834709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:53.834709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:53.834709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:53.834709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:53.834709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.834709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:53.834709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:53.834709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:53.834709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.835154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:53.835154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:53.835154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:53.835154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:53.835154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:53.835154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:53.835154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:53.835154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:53.835154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.835154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:53.835154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:53.835154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.835154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:53.835610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:53.835979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.836420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:53.836420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:53.836420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.836420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:53.836420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.836420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:53.839191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:53.839521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:53.839689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:53.839810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:52:53.840288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:53.840479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.841023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:53.841023 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.841023 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:53.841023 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:53.841023 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.841023 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:53.841023 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.841023 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:53.841618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:53.842183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:53.842183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:53.842183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.842183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:53.842183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:53.842183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.842183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:53.842183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.842183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:53.842593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.842593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:53.842593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:53.842593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:53.842593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:53.842593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:53.842593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:53.842593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:52:53.842593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:53.842593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:53.842593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.842593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:53.843031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:53.844167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:53.844632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:53.845062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:53.845062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:53.845062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:53.845062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:53.845062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:53.845062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.845062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:53.845062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:53.845062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.845062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:53.845062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.845062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:53.846075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:53.846360 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:53.846439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:53.846552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:53.846993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:53.847217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.847854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:53.847854 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.847854 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:53.847854 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:53.847854 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.847854 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:53.847854 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.847854 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:53.849392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:53.850011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:53.850472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:53.850472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:53.850472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:53.850472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:53.850472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.850472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:53.850472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:53.850472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:53.850472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.850472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:53.850472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:53.850472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:53.850472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:53.850827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:53.850827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:53.850827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:53.850827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:53.850827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.850827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:53.850827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:53.850827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.850827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:53.850827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.850827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:53.850827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:53.851177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.851702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:53.852533 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:53.852533 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:53.852533 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:53.852533 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:53.852736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:53.852950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:53.853079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.853613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:53.853613 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.853613 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:53.853613 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:53.853613 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.853613 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:53.853613 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.853613 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:53.854275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.854275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:53.854275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:53.854275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:53.854275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:53.854275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:53.854275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:53.854275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.854275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:53.854275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:53.854275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:53.854275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.854637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:53.854637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:53.854637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:53.854637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:53.854637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:53.854637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:53.854637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:53.854637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:53.854637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.854637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:53.854637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:53.854637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.854637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:53.854980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:53.855393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:53.855903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:53.855903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:53.855903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:53.855903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:53.855903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:53.855903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:56.139109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:56.139716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:56.139797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:56.140664 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:56.176649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:56.176927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:56.177681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:56.177681 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:56.177681 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:56.177681 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:56.177681 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:56.177681 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:56.177681 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:56.177681 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:56.178628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:56.178628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:56.178628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:56.178628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:56.178628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:56.178628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:56.178628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:56.178628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:56.178628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:56.178628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:56.178628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:56.178628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:56.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:56.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:56.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:56.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:56.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:56.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:56.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:56.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:56.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:56.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:56.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:56.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:56.178969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:56.179269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:56.179269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:56.179513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:56.180047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:56.180543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:56.180543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:56.180543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:58.830242 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:58.831116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:58.831116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:58.831931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:58.832320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:58.832605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.833541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:58.833541 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.833541 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:58.833541 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:58.833541 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.833541 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:58.833541 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.833541 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:58.834874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.834874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:58.834874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:58.834874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:58.834874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:58.834874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:58.834874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:58.834874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.834874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:58.834874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:58.834874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:58.834874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.835792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:58.835792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:58.835792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:58.835792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:58.835792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:58.835792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:58.835792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:58.835792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:58.835792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.835792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:58.835792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:58.835792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.835792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:58.836408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:58.837048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.837883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:58.837883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:58.837883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.837883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:58.837883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.837883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:58.839601 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:58.839998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:58.840399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:58.840927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:52:58.841515 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:58.841804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.842845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:58.842845 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.842845 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:58.842845 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:58.842845 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.842845 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:58.842845 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.842845 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:58.843948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:58.844605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:58.844605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:58.844605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.844605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:58.844605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:58.844605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.844605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:58.844605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.844605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:58.844991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.844991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:58.844991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:58.844991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:58.844991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:58.844991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:58.844991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:58.844991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:52:58.844991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:58.844991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:58.844991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.844991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:58.845528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:58.846126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.846126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:58.846126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.846126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:58.846337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:58.847022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:58.847830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.847830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:58.847830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:58.847830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.847830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:58.847830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.847830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:58.847830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:58.847830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:58.847830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:58.848411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:58.848411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:58.849010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.849855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:58.849855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.849855 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:58.849855 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:58.849855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.849855 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:58.849855 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.849855 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:52:58.850823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:52:58.851378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:58.851378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:52:58.851378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.851378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:52:58.851378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:52:58.851378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.851378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:52:58.851378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.851378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:52:58.851870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.851870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:58.851870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:58.851870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:58.851870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:58.851870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:58.851870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:58.851870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:58.851870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:58.851870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.851870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:58.852334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:58.852951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.852951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:58.852951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.852951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:58.853307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:58.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:58.854606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.854606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:58.854606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.854606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:52:58.854606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:52:58.854606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:52:58.854606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:52:58.854606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:52:58.854949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:58.854949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:52:58.855044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.855658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:52:58.855658 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.855658 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:52:58.855658 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:52:58.855658 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.855658 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:52:58.855658 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.855658 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:52:58.856595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.856595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:52:58.856595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:58.856595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:52:58.856595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:52:58.856595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:58.856595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:58.856595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.856595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:58.856595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:58.856595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:52:58.856595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.856974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:52:58.856974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:58.856974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:52:58.856974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:58.856974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:52:58.856974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:52:58.856974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:58.856974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:52:58.856974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.856974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:52:58.856974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:52:58.856974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.856974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:52:58.857406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.857406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:52:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:52:58.858095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:52:58.858672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:52:58.858672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:52:58.858672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:01.139776 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:01.140568 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:01.140670 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:01.142529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:01.177872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:01.178130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:01.178940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:01.178940 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:01.178940 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:01.178940 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:01.178940 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:01.178940 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:01.178940 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:01.178940 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:01.180565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:01.180565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:01.180565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:01.180565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:01.180565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:01.180565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:01.180565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:01.180565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:01.180565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:01.180565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:01.180565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:01.180565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:01.181123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:01.181123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:01.181123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:01.181123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:01.181123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:01.181123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:01.181123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:01.181123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:01.181123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:01.181123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:01.181123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:01.181123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:01.181123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:01.181707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:01.181707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:01.184285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:01.184716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:01.185108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:01.185108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:01.185108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:03.830899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:03.831840 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:03.831840 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:03.832665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:03.832994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:03.833393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.834203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:03.834203 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.834203 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:03.834203 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:03.834203 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.834203 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:03.834203 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.834203 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:03.836275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.836275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:03.836275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:03.836275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:03.836275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:03.836275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:03.836275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:03.836275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.836275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:03.836275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:03.836275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:03.836275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.837381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:03.837381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:03.837381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:03.837381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:03.837381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:03.837381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:03.837381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:03.837381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:03.837381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.837381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:03.837381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:03.837381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.837381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:03.838250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:03.838900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.841954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:03.841954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:03.841954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.841954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:03.841954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.841954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:03.841954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:03.841954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:03.841954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:03.841954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:03.842458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.842978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:03.843735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.843735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:03.843735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.843735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:03.843735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:03.843735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:03.843735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:03.843735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:03.843735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:03.843735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:53:03.843735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:03.843735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:03.844195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:03.846600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.847138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:03.847845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:03.848406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:03.848406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:03.848406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:03.848406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:03.848406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.848642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:03.848642 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.848642 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:03.848642 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:03.848642 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.848642 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:03.848642 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.848642 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:03.849486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:03.850038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:03.850038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:03.850038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.850038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:03.850038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:03.850038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.850038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:03.850038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.850038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:03.850341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.850341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:03.850341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:03.850341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:03.850341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:03.850341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:03.850341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:03.850341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:03.850341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:03.850341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.850341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:03.850636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:03.850967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.850967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:03.850967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.850967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:03.851372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:03.851820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:03.852276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.852276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:03.852276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.852276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:03.854059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:03.854338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:03.854445 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:03.854640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:03.854945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:03.855155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:03.855318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.855786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:03.855786 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.855786 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:03.855786 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:03.855786 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.855786 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:03.855786 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.855786 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:03.856620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.856620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:03.856620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:03.856620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:03.856620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:03.856620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:03.856620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:03.856620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.856620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:03.856620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:03.856620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:03.856620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.857033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:03.857033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:03.857033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:03.857033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:03.857033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:03.857033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:03.857033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:03.857033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:03.857033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.857033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:03.857033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:03.857033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.857033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:03.857467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.857467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:03.857535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:03.857981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:03.858493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:03.858493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:03.858493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:06.140426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:06.141273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:06.141273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:06.142009 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:06.179904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:06.180288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:06.181322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:06.181322 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:06.181322 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:06.181322 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:06.181322 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:06.181322 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:06.181322 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:06.181322 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:06.182194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:06.182194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:06.182194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:06.182194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:06.182194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:06.182194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:06.182194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:06.182194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:06.182194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:06.182194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:06.182194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:06.182194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:06.182621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:06.182621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:06.182621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:06.182621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:06.182621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:06.182621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:06.182621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:06.182621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:06.182621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:06.182621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:06.182621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:06.182621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:06.182621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:06.182951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:06.183604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:06.184282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:06.184282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:06.184282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:06.184282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:06.184282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:06.184282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:08.831637 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:08.832333 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:08.832333 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:08.833093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:08.833505 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:08.833814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.834784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:08.834784 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.834784 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:08.834784 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:08.834784 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.834784 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:08.834784 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.834784 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:08.835982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.835982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:08.835982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:08.835982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:08.835982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:08.835982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:08.835982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:08.835982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.835982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:08.835982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:08.835982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:08.835982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.836740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:08.836740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:08.836740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:08.836740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:08.836740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:08.836740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:08.836740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:08.836740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:08.836740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.836740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:08.836740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:08.836740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.836740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:08.837479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:08.839229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.840983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:08.840983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:08.840983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.840983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:08.840983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.840983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:08.840983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:08.842257 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:08.842390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:08.842621 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:53:08.843193 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:08.843548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.844428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:08.844428 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.844428 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:08.844428 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:08.844428 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.844428 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:08.844428 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.844428 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:08.845568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:08.846315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:08.846315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:08.846315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.846315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:08.846315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:08.846315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.846315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:08.846315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.846315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:08.846822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.846822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:08.846822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:08.846822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:08.846822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:08.846822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:08.846822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:08.846822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:53:08.846822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:08.846822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:08.846822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.846822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:08.848133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:08.849192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:08.849788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:08.850577 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:08.851251 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:08.851251 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:08.851251 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:08.851251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.851463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:08.851463 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.851463 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:08.851463 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:08.851463 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.851463 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:08.851463 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.851463 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:08.852540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:08.853075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:08.853075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:08.853075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.853075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:08.853075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:08.853075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.853075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:08.853075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.853075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:08.853627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.853627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:08.853627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:08.853627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:08.853627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:08.853627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:08.853627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:08.853627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:08.853627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:08.853627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.853627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:08.854076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:08.854669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.854669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:08.854669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.854669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:08.855265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:08.856089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:08.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:08.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:08.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:08.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:08.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:08.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:08.857309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:08.857309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:08.857309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.857851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:08.857851 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.857851 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:08.857851 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:08.857851 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.857851 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:08.857851 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.857851 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:08.858944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.858944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:08.858944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:08.858944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:08.858944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:08.858944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:08.858944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:08.858944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.858944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:08.858944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:08.858944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:08.858944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.859497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:08.859497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:08.859497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:08.859497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:08.859497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:08.859497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:08.859497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:08.859497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:08.859497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.859497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:08.859497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:08.859497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.859497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:08.860075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:08.860867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:08.861851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:08.861851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:08.861851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:08.861851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:08.861851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:08.861851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:11.141161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:11.142857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:11.142857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:11.143523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:11.145438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=260 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:53:11.177735 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:11.178162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:11.179011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:11.179011 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:11.179011 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:11.179011 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:11.179011 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:11.179011 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:11.179011 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:11.179011 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:11.180087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:11.180087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:11.180087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:11.180087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:11.180087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:11.180087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:11.180087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:11.180087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:11.180087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:11.180087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:11.180087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:11.180087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:11.180752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:11.180752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:11.180752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:11.180752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:11.180752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:11.180752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:11.180752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:11.180752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:11.180752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:11.180752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:11.180752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:11.180752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:11.180752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:11.181373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:11.182099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:11.183006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:11.183006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:11.183006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:11.183006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:11.183006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:11.183006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:13.832362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:13.833428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:13.833428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:13.834126 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:13.834488 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:13.834729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.835745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:13.835745 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.835745 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:13.835745 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:13.835745 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.835745 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:13.835745 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.835745 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:13.837198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.837198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:13.837198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:13.837198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:13.837198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:13.837198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:13.837198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:13.837198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.837198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:13.837198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:13.837198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:13.837198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.837658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:13.837658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:13.837658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:13.837658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:13.837658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:13.837658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:13.837658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:13.837658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:13.837658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.837658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:13.837658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:13.837658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.837658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:13.838081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.838081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:13.839056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:13.839642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.840299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:13.840299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.840299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:13.841982 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:13.842280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:13.842388 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:13.842514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:53:13.842927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:13.843224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.843882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:13.843882 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.843882 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:13.843882 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:13.843882 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.843882 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:13.843882 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.843882 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:13.844675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:13.845185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:13.845185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:13.845185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.845185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:13.845185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:13.845185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.845185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:13.845185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.845185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:13.845508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.845508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:13.845508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:13.845508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:13.845508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:13.845508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:13.845508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:13.845508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:53:13.845508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:13.845508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:13.845508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.845508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:13.845887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:13.846299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.846299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:13.846299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.846299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:13.846607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:13.847101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:13.847665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.847665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:13.847665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:13.847665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.847665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:13.847665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.847665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:13.849028 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:13.849309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:13.849426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:13.849553 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:13.849921 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:13.850287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.850988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:13.850988 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.850988 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:13.850988 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:13.850988 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.850988 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:13.850988 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.850988 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:13.851743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:13.852167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:13.852167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:13.852167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.852167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:13.852167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:13.852167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.852167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:13.852167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.852167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:13.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:13.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:13.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:13.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:13.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:13.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:13.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:13.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:13.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:13.853001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:13.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:13.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:13.854115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:13.854730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.854730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:13.854730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.854730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:13.855831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:13.856085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:13.856158 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:13.856339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:13.856690 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:13.857014 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:13.857232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.857877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:13.857877 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.857877 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:13.857877 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:13.857877 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.857877 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:13.857877 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.857877 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:13.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:13.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:13.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:13.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:13.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:13.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:13.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:13.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:13.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:13.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.859076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:13.859076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:13.859076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:13.859076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:13.859076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:13.859076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:13.859076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:13.859076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:13.859076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.859076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:13.859076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:13.859076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.859076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:13.859516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.859516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:13.859650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:13.860160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:13.861016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:13.861016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:13.861016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:16.141805 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:16.142673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:16.142673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:16.143837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:16.177654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:16.178408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:16.179233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:16.179233 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:16.179233 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:16.179233 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:16.179233 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:16.179233 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:16.179233 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:16.179233 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:16.180127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:16.180127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:16.180127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:16.180127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:16.180127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:16.180127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:16.180127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:16.180127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:16.180127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:16.180127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:16.180127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:16.180127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:16.180793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:16.180793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:16.180793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:16.180793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:16.180793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:16.180793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:16.180793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:16.180793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:16.180793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:16.180793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:16.180793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:16.180793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:16.180793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:16.181460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:16.182369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:16.182938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:16.182938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:16.182938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:16.182938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:16.182938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:16.182938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:18.832996 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:18.833685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:18.833926 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:18.834890 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:18.835160 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:18.835434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.836332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:18.836332 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.836332 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:18.836332 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:18.836332 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.836332 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:18.836332 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.836332 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:18.837598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.837598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:18.837598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:18.837598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:18.837598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:18.837598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:18.837598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:18.837598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.837598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:18.837598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:18.837598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:18.837598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.837900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:18.837900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:18.837900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:18.837900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:18.837900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:18.837900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:18.837900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:18.837900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:18.837900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.837900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:18.837900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:18.837900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.837900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:18.838175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.838175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:18.838804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:18.839164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.839557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:18.842423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:18.842685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:18.842770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:18.842859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:53:18.843294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:18.843573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.844216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:18.844216 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.844216 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:18.844216 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:18.844216 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.844216 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:18.844216 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.844216 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:18.845169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:18.845631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:18.845631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:18.845631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.845631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:18.845631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:18.845631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.845631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:18.845631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.845631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:18.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:18.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:18.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:18.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:18.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:18.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:18.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:53:18.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:18.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:18.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.846098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:18.846531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:18.847034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.847034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:18.847034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.847034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:18.847640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:18.848086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:18.848586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.848586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:18.848586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:18.848586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.848586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:18.848586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.848586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:18.849417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:18.849716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:18.849831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:18.849963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:18.850312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:18.850525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.852427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:18.852427 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.852427 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:18.852427 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:18.852427 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.852427 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:18.852427 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.852427 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:18.853261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:18.853703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:18.853703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:18.853703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.853703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:18.853703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:18.853703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.853703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:18.853703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.853703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:18.854143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.854143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:18.854143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:18.854143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:18.854143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:18.854143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:18.854143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:18.854143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:18.854143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:18.854143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.854143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:18.854449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:18.854827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.854827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:18.854827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.854827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:18.854969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:18.855419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:18.855942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.855942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:18.855942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.855942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:18.856286 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:18.856585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:18.856668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:18.856787 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:18.857088 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:18.857441 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:18.857669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.858173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:18.858173 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.858173 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:18.858173 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:18.858173 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.858173 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:18.858173 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.858173 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:18.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:18.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:18.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:18.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:18.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:18.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:18.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:18.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:18.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:18.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.859143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:18.859143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:18.859143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:18.859143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:18.859143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:18.859143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:18.859143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:18.859143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:18.859143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.859143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:18.859143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:18.859143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.859143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:18.859520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.859520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:18.859615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:18.860136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:18.860634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:18.860634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:18.860634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:21.142425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:21.143162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:21.143162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:21.143843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:21.179070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:21.179506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:21.180373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:21.180373 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:21.180373 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:21.180373 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:21.180373 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:21.180373 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:21.180373 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:21.180373 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:21.181445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:21.181445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:21.181445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:21.181445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:21.181445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:21.181445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:21.181445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:21.181445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:21.181445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:21.181445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:21.181445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:21.181445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:21.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:21.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:21.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:21.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:21.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:21.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:21.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:21.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:21.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:21.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:21.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:21.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:21.182068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:21.182692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:21.183422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:21.184354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:21.184354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:21.184354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:21.184354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:21.184354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:21.184354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:23.833756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:23.834863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:23.834863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:23.835452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:23.835709 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:23.835860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.836595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:23.836595 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.836595 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:23.836595 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:23.836595 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.836595 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:23.836595 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.836595 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:23.838095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.838095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:23.838095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:23.838095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:23.838095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:23.838095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:23.838095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:23.838095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.838095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:23.838095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:23.838095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:23.838095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.838863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:23.838863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:23.838863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:23.838863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:23.838863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:23.838863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:23.838863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:23.838863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:23.838863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.838863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:23.838863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:23.838863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.838863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:23.839587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.839587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:23.839783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:23.840667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.841392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:23.841392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.841392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:23.842771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:23.843148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:23.843364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:23.843555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:53:23.843921 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:23.844120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.844676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:23.844676 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.844676 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:23.844676 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:23.844676 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.844676 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:23.844676 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.844676 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:23.845416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:23.845972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:23.845972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:23.845972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.845972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:23.845972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:23.845972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.845972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:23.845972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.845972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:23.846323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.846323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:23.846323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:23.846323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:23.846323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:23.846323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:23.846323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:23.846323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:53:23.846323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:23.846323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:23.846323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.846323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:23.846735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:23.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:23.847755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:23.848441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:23.848441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:23.848441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:23.848441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:23.848441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:23.848441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.848441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:23.848441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:23.848441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.848441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:23.848441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.848441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:23.849744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:23.850000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:23.850098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:23.850240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:23.850560 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:23.850780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.851251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:23.851251 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.851251 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:23.851251 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:23.851251 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.851251 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:23.851251 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.851251 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:23.851967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:23.852446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:23.852446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:23.852446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.852446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:23.852446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:23.852446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.852446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:23.852446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.852446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:23.852808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.852808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:23.852808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:23.852808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:23.852808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:23.852808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:23.852808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:23.852808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:23.852808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:23.852808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.852808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:23.853199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:23.853640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.853640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:23.853640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.853640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:23.853863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:23.854430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:23.854989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.854989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:23.854989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.854989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:23.856700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:23.856958 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:23.857068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:23.857229 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:23.857524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:23.857743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:23.857877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.858410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:23.858410 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.858410 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:23.858410 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:23.858410 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.858410 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:23.858410 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.858410 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:23.859251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.859251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:23.859251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:23.859251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:23.859251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:23.859251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:23.859251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:23.859251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.859251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:23.859251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:23.859251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:23.859251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.859659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:23.859659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:23.859659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:23.859659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:23.859659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:23.859659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:23.859659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:23.859659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:23.859659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.859659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:23.859659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:23.859659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.859659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:23.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:23.860311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:23.860832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:23.861408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:23.861408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:23.861408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:26.143092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:26.143956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:26.143956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:26.145104 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:26.177504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:26.177977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:26.178855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:26.178855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:26.178855 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:26.178855 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:26.178855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:26.178855 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:26.178855 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:26.178855 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:26.179613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:26.179613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:26.179613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:26.179613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:26.179613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:26.179613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:26.179613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:26.179613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:26.179613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:26.179613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:26.179613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:26.179613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:26.180258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:26.180258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:26.180258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:26.180258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:26.180258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:26.180258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:26.180258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:26.180258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:26.180258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:26.180258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:26.180258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:26.180258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:26.180258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:26.180939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:26.181654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:26.182644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:26.182644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:26.182644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:26.182644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:26.182644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:26.182644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:28.835336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:28.835336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:28.838880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:28.838880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:28.838880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:28.838880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.838880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:28.838880 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.838880 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:28.838880 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:28.838880 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.838880 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:28.838880 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.838880 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:28.840800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.840800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:28.840800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:28.840800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:28.840800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:28.840800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:28.840800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:28.840800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.840800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:28.840800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:28.840800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:28.840800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.843195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:28.843195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:28.843195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:28.843195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:28.843195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:28.843195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:28.843195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:28.843195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:28.843195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.843195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:28.843195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:28.843195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.843195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:28.844628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:28.846638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.848116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:28.848116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:28.848116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.848116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:28.848116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.848116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:28.848116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:28.848116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:28.848116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:28.848116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:28.848398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.848679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:28.849002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.849002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:28.849002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.849002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:28.849002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:28.849002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:28.849002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:28.849002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:28.849002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:28.849002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:53:28.849002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:28.849002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:28.849938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:28.850580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.851035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:28.851724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:28.852541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:28.852541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:28.852541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:28.852541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:28.852541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.852541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:28.852541 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.852541 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:28.852541 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:28.852717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:28.852984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:28.852984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:28.852984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:28.852984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:28.852984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:28.852984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.852984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:28.852984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:28.852984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.852984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:28.852984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.852984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:28.853196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.853196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:28.853196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:28.853196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:28.853196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:28.853196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:28.853196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:28.853196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:28.853196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:28.853196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.853196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:28.853400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:28.853642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.853642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:28.853642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.853642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:28.855165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:28.856283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:28.857093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.857093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:28.857093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.857093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:28.857093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:28.857293 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:28.857417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:28.857567 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:28.857891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:28.858133 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:28.858294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.858794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:28.858794 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.858794 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:28.858794 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:28.858794 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.858794 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:28.858794 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.858794 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:28.859680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.859680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:28.859680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:28.859680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:28.859680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:28.859680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:28.859680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:28.859680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.859680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:28.859680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:28.859680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:28.859680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.860041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:28.860041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:28.860041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:28.860041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:28.860041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:28.860041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:28.860041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:28.860041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:28.860041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.860041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:28.860041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:28.860041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.860041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:28.860340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.860340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:28.860752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:28.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:28.861567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:28.861567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:28.861567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:31.143989 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:31.145047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:31.145047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:31.145847 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:31.180124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:31.180589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:31.181488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:31.181488 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:31.181488 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:31.181488 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:31.181488 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:31.181488 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:31.181488 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:31.181488 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:31.182611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:31.182611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:31.182611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:31.182611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:31.182611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:31.182611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:31.182611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:31.182611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:31.182611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:31.182611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:31.182611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:31.182611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:31.183279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:31.183279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:31.183279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:31.183279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:31.183279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:31.183279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:31.183279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:31.183279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:31.183279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:31.183279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:31.183279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:31.183279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:31.183279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:31.183862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:31.184505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:31.185259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:31.185259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:31.185259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:31.185259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:31.185259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:31.185259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:33.835504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:33.836188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:33.836188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:33.837355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:33.837355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:33.837630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.838234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:33.838234 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.838234 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:33.838234 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:33.838234 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.838234 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:33.838234 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.838234 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:33.839753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.839753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:33.839753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:33.839753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:33.839753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:33.839753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:33.839753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:33.839753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.839753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:33.839753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:33.839753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:33.839753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.840331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:33.840331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:33.840331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:33.840331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:33.840331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:33.840331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:33.840331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:33.840331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:33.840331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.840331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:33.840331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:33.840331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.840331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:33.840852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.840852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:33.841474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:33.842166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.842997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:33.842997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.842997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:33.843409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:33.843658 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:33.843758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:33.843896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:53:33.844180 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:33.844400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.844881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:33.844881 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.844881 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:33.844881 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:33.844881 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.844881 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:33.844881 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.844881 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:33.845647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:33.846293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:33.846293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:33.846293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.846293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:33.846293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:33.846293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.846293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:33.846293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.846293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:33.846564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.846564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:33.846564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:33.846564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:33.846564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:33.846564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:33.846564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:33.846564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:53:33.846564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:33.846564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:33.846564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.846564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:33.846894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:33.847294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.847294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:33.847294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.847294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:33.847428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:33.847873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:33.848394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.848394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:33.848394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:33.848394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.848394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:33.848394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.848394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:33.850480 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:33.850740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:33.850850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:33.850972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:33.851391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:33.851731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:33.852598 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.852598 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:33.852598 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:33.852598 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.852598 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:33.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.852598 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:33.853355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:33.853743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:33.853743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:33.853743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.853743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:33.853743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:33.853743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.853743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:33.853743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.853743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:33.854196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.854196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:33.854196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:33.854196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:33.854196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:33.854196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:33.854196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:33.854196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:33.854196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:33.854196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.854196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:33.854549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:33.854910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.854910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:33.854910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.854910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:33.855224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:33.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:33.856129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.856129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:33.856129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.856129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:33.857424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:33.857687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:33.857795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:33.857940 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:33.858340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:33.858673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:33.858897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.859569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:33.859569 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.859569 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:33.859569 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:33.859569 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.859569 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:33.859569 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.859569 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:33.860374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.860374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:33.860374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:33.860374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:33.860374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:33.860374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:33.860374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:33.860374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.860374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:33.860374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:33.860374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:33.860374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.860710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:33.860710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:33.860710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:33.860710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:33.860710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:33.860710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:33.860710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:33.860710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:33.860710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.860710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:33.860710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:33.860710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.860710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:33.861041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.861041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:33.861366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:33.861787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:33.862264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:33.862264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:33.862264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:36.144742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:36.145308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:36.145512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:36.146746 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:36.173878 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:36.174145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:36.174750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:36.174750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:36.174750 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:36.174750 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:36.174750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:36.174750 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:36.174750 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:36.174750 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:36.175465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:36.175465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:36.175465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:36.175465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:36.175465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:36.175465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:36.175465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:36.175465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:36.175465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:36.175465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:36.175465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:36.175465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:36.175727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:36.175727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:36.175727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:36.175727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:36.175727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:36.175727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:36.175727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:36.175727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:36.175727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:36.175727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:36.175727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:36.175727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:36.175727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:36.175975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:36.175975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:36.176197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:36.176529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:36.176898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:36.176898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:36.176898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:38.836064 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:38.836748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:38.836883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:38.837760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:38.837998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:38.838226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.838902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:38.838902 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.838902 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:38.838902 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:38.838902 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.838902 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:38.838902 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.838902 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:38.840456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.840456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:38.840456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:38.840456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:38.840456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:38.840456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:38.840456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:38.840456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.840456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:38.840456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:38.840456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:38.840456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.840960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:38.840960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:38.840960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:38.840960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:38.840960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:38.840960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:38.840960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:38.840960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:38.840960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.840960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:38.840960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:38.840960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.840960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:38.842413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:38.843407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.844178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:38.844178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:38.844178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.844178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:38.844178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.844178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:38.844178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:38.844178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:38.844178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:38.845023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:53:38.845023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:38.845023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.845247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:38.845247 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.845247 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:38.845247 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:38.845247 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.845247 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:38.845247 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.845247 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:38.846014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:38.846971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:38.847343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:38.847343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:38.847343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:53:38.847343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:38.847343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:38.847343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.847343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:38.847343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:38.847343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:38.847343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.847343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:38.847343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:38.847343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:38.847711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:38.847711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:38.847711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:38.847711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:38.847711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:38.847711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.847711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:38.847711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:38.847711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.847711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:38.847711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.847711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:38.847998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:38.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:38.848849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.848849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:38.848849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:38.848849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.848849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:38.848849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.848849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:38.850947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:38.851273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:38.851391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:38.851555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:38.852047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:38.852432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.853121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:38.853121 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.853121 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:38.853121 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:38.853121 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.853121 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:38.853121 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.853121 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:38.853976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:38.854375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:38.854375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:38.854375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.854375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:38.854375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:38.854375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.854375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:38.854375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.854375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:38.854954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.854954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:38.854954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:38.854954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:38.854954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:38.854954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:38.854954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:38.854954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:38.854954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:38.854954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.854954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:38.855268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:38.855631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.855631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:38.855631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.855631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:38.856139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:38.856592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:38.857026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.857026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:38.857026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.857026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:38.857765 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:38.858005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:38.858155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:38.858318 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:38.858596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:38.858807 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:38.858992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:38.860465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.860465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:38.860465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:38.860465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:38.860465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:38.860465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:38.860465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:38.860465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.860465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:38.860465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:38.860465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:38.860465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.860767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:38.860767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:38.860767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:38.860767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:38.860767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:38.860767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:38.860767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:38.860767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:38.860767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.860767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:38.860767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:38.860767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.860767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:38.861083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.861083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:38.861479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:38.861902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:38.862529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:38.862529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:38.862529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:41.145491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:41.146409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:41.146409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:41.147413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:41.180403 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:41.180403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:41.181009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:41.181009 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:41.181009 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:41.181009 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:41.181009 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:41.181009 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:41.181009 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:41.181009 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:41.181829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:41.181829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:41.181829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:41.181829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:41.181829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:41.181829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:41.181829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:41.181829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:41.181829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:41.181829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:41.181829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:41.181829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:41.182571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:41.182571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:41.182571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:41.182571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:41.182571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:41.182571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:41.182571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:41.182571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:41.182571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:41.182571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:41.182571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:41.182571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:41.182571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:41.183226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:41.184017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:41.185013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:41.185013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:41.185013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:41.185013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:41.185013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:41.185013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:43.836943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:43.839147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:43.839147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:43.839147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:43.839369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:43.839604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.840601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:43.840601 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.840601 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:43.840601 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:43.840601 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.840601 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:43.840601 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.840601 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:43.841760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.841760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:43.841760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:43.841760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:43.841760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:43.841760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:43.841760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:43.841760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.841760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:43.841760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:43.841760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:43.841760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.842337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:43.842337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:43.842337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:43.842337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:43.842337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:43.842337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:43.842337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:43.842337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:43.842337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.842337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:43.842337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:43.842337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.842337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:43.842860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.842860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:43.843072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:43.843734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.844563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:43.844563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.844563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:43.844563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:43.844563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:43.844563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:43.844868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:53:43.845001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:43.845251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.845769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:43.845769 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.845769 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:43.845769 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:43.845769 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.845769 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:43.845769 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.845769 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:43.846771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:43.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:43.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:43.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:43.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:43.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:43.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.847292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:43.847618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.847618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:43.847618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:43.847618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:43.847618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:43.847618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:43.847618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:43.847618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:53:43.847618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:43.847618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:43.847618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.847618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:43.848001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:43.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:43.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.848407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:43.848852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:43.849343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:43.849873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.849873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:43.849873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:43.849873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.849873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:43.849873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.849873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:43.851364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:43.851669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:43.851783 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:43.851918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:43.852306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:43.852541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.853093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:43.853093 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.853093 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:43.853093 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:43.853093 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.853093 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:43.853093 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.853093 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:43.853908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:43.854337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:43.854337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:43.854337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.854337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:43.854337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:43.854337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.854337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:43.854337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.854337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:43.854857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.854857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:43.854857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:43.854857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:43.854857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:43.854857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:43.854857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:43.854857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:43.854857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:43.854857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.854857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:43.855248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:43.855640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.855640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:43.855640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.855640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:43.855975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:43.856484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:43.856994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.856994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:43.856994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.856994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:43.858122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:43.858407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:43.858522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:43.858689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:43.859034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:43.859291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:43.859445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.860025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:43.860025 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.860025 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:43.860025 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:43.860025 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.860025 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:43.860025 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.860025 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:43.860928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.860928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:43.860928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:43.860928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:43.860928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:43.860928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:43.860928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:43.860928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.860928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:43.860928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:43.860928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:43.860928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.861292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:43.861292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:43.861292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:43.861292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:43.861292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:43.861292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:43.861292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:43.861292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:43.861292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.861292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:43.861292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:43.861292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.861292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:43.861812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.861812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:43.862165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:43.862639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:43.863304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:43.863304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:43.863304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:46.146043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:46.146905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:46.146905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:46.148162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:46.188183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:46.188620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:46.189499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:46.189499 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:46.189499 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:46.189499 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:46.189499 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:46.189499 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:46.189499 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:46.189499 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:46.190539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:46.190539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:46.190539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:46.190539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:46.190539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:46.190539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:46.190539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:46.190539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:46.190539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:46.190539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:46.190539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:46.190539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:46.191101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:46.191101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:46.191101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:46.191101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:46.191101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:46.191101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:46.191101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:46.191101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:46.191101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:46.191101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:46.191101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:46.191101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:46.191101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:46.191696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:46.191696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:46.191793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:46.192443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:46.193253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:46.193253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:46.193253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:48.837956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:48.839240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:48.839240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:48.841946 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:48.842341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:48.842580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.843522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:48.843522 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.843522 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:48.843522 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:48.843522 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.843522 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:48.843522 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.843522 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:48.845939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.845939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:48.845939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:48.845939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:48.845939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:48.845939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:48.845939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:48.845939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.845939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:48.845939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:48.845939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:48.845939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.846426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:48.846426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:48.846426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:48.846426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:48.846426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:48.846426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:48.846426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:48.846426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:48.846426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.846426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:48.846426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:48.846426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.846426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:48.846811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.846811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:48.846811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:48.848515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:48.849057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.849716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:48.849716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.849716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:48.849716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:48.849716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:48.849716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:53:48.850018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:48.850229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.850949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:48.850949 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.850949 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:48.850949 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:48.850949 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.850949 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:48.850949 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.850949 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:48.852410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:48.852956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:48.852956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:48.852956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.852956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:48.852956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:48.852956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.852956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:48.852956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.852956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:48.852956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:48.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:48.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:48.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:48.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:48.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:48.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:48.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:53:48.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:48.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:48.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.853408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:48.853778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:48.854175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:48.854618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:48.855160 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:48.855619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:48.855619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:48.855619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.855998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:48.855998 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.855998 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:48.855998 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:48.855998 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.855998 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:48.855998 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.855998 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:48.856567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:48.857054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:48.857054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:48.857054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.857054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:48.857054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:48.857054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.857054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:48.857054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.857054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:48.857353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.857353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:48.857353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:48.857353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:48.857353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:48.857353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:48.857353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:48.857353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:48.857353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:48.857353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.857353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:48.857702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:48.858151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:48.858609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:48.859181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:48.859181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:48.859181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:48.859181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:48.859181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.859181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:48.859181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:48.859181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.859181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:48.859181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.859181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:48.859181 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:48.859181 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:48.859583 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:48.859583 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:48.859583 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:48.859684 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:48.859922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.860844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:48.860844 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.860844 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:48.860844 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:48.860844 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.860844 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:48.860844 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.860844 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:48.861671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.861671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:48.861671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:48.861671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:48.861671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:48.861671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:48.861671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:48.861671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.861671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:48.861671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:48.861671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:48.861671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.862044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:48.862044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:48.862044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:48.862044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:48.862044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:48.862044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:48.862044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:48.862044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:48.862044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.862044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:48.862044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:48.862044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.862044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:48.862430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.862430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:48.862783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:48.863226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:48.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:48.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:48.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:51.146688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:51.147612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:51.147612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:51.148563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:51.196780 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:51.196997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:51.197455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:51.197455 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:51.197455 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:51.197455 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:51.197455 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:51.197455 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:51.197455 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:51.197455 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:51.198089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:51.198089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:51.198089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:51.198089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:51.198089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:51.198089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:51.198089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:51.198089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:51.198089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:51.198089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:51.198089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:51.198089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:51.198763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:51.198763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:51.198763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:51.198763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:51.198763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:51.198763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:51.198763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:51.198763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:51.198763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:51.198763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:51.198763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:51.198763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:51.198763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:51.199416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:51.200263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:51.200941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:51.200941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:51.200941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:51.200941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:51.200941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:51.200941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:53.838752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:53.839369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:53.839419 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:53.840499 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:53.840938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:53.841107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.842344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:53.842344 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.842344 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:53.842344 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:53.842344 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.842344 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:53.842344 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.842344 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:53.843513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.843513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:53.843513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:53.843513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:53.843513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:53.843513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:53.843513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:53.843513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.843513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:53.843513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:53.843513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:53.843513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.843810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:53.843810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:53.843810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:53.843810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:53.843810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:53.843810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:53.843810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:53.843810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:53.843810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.843810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:53.843810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:53.843810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.843810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:53.844065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.844065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:53.845124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:53.845475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.845923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:53.845923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.845923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:53.848985 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:53.849350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:53.849417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:53.849549 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:53:53.849978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:53.850231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.850922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:53.850922 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.850922 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:53.850922 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:53.850922 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.850922 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:53.850922 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.850922 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:53.851748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:53.852247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:53.852247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:53.852247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.852247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:53.852247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:53.852247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.852247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:53.852247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.852247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:53.852661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.852661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:53.852661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:53.852661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:53.852661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:53.852661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:53.852661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:53.852661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:53:53.852661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:53.852661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:53.852661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.852661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:53.853078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:53.853477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.853477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:53.853477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.853477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:53.854007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:53.854448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:53.854939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.854939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:53.854939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:53.854939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.854939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:53.854939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.854939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:53.854939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:53.855348 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:53.855409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:53.855563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:53.855920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:53.856232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.856863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:53.856863 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.856863 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:53.856863 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:53.856863 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.856863 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:53.856863 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.856863 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:53.857664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:53.858634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:53.859008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:53.859008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:53.859008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:53.859008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:53.859008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.859008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:53.859008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:53.859008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:53.859008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.859008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:53.859008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:53.859008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:53.859008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:53.859428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:53.859428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:53.859428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:53.859428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:53.859428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.859428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:53.859428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:53.859428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.859428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:53.859428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.859428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:53.859813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:53.860165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:53.860540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.860540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:53.860540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.860540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:53.860540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:53.860540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:53.860540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:53.860540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:53.860878 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:53.861351 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:53.861565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.862095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:53.862095 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.862095 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:53.862095 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:53.862095 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.862095 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:53.862095 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.862095 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:53.862965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.862965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:53.862965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:53.862965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:53.862965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:53.862965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:53.862965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:53.862965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.862965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:53.862965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:53.862965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:53.862965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.863357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:53.863357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:53.863357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:53.863357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:53.863357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:53.863357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:53.863357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:53.863357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:53.863357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.863357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:53.863357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:53.863357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.863357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:53.863603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.863603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:53.863971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:53.864473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:53.864896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:53.864896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:53.864896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:56.147344 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:56.148056 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:56.148056 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:56.149353 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:56.184062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:56.184347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:56.184983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:56.184983 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:56.184983 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:56.184983 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:56.184983 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:56.184983 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:56.184983 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:56.184983 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:56.185751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:56.185751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:56.185751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:56.185751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:56.185751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:56.185751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:56.185751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:56.185751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:56.185751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:56.185751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:56.185751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:56.185751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:56.186104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:56.186104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:56.186104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:56.186104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:56.186104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:56.186104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:56.186104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:56.186104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:56.186104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:56.186104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:56.186104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:56.186104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:56.186104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:56.186478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:56.186478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:56.186544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:56.186966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:56.187485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:56.187485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:56.187485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:58.839440 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:58.840075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:58.840168 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:58.840918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:58.841118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:58.841249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.841716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:58.841716 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.841716 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:58.841716 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:58.841716 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.841716 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:58.841716 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.841716 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:58.842503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.842503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:58.842503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:58.842503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:58.842503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:58.842503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:58.842503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:58.842503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.842503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:58.842503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:58.842503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:58.842503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.842768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:58.842768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:58.842768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:58.842768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:58.842768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:58.842768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:58.842768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:58.842768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:58.842768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.842768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:58.842768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:58.842768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.842768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:58.842994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.842994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:58.843368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:58.843668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.844023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:58.844023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.844023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:58.849450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:58.849725 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:58.849838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:58.849956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:53:58.850376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:58.850588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.851094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:58.851094 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.851094 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:58.851094 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:58.851094 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.851094 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:58.851094 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.851094 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:58.852012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:58.852323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:58.852323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:58.852323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.852323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:58.852323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:58.852323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.852323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:58.852323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.852323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:58.853470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.853470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:58.853470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:58.853470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:58.853470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:58.853470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:58.853470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:58.853470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:53:58.853470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:58.853470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:58.853470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.853470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:58.853720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:58.853968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.853968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:58.853968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.853968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:58.855272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:58.855657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:58.855972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.855972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:58.855972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:58.855972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.855972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:58.855972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.855972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:58.855972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:58.855972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:58.856143 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:58.856307 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:58.857063 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:58.857632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.858679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:58.858679 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.858679 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:58.858679 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:58.858679 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.858679 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:58.858679 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.858679 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:53:58.859854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:53:58.860150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:58.860150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:53:58.860150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.860150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:53:58.860150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:53:58.860150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.860150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:53:58.860150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.860150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:53:58.860733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.860733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:58.860733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:58.860733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:58.860733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:58.860733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:58.860733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:58.860733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:58.860733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:58.860733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.860733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:58.860981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:58.861264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.861264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:58.861264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.861264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:58.861896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:58.862270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:58.862642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.862642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:58.862642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.862642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:53:58.862642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:53:58.862642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:53:58.862642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:53:58.862642 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:53:58.862987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:58.863175 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:53:58.863404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.863921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:53:58.863921 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.863921 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:53:58.863921 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:53:58.863921 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.863921 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:53:58.863921 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.863921 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:53:58.864725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.864725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:53:58.864725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:58.864725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:53:58.864725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:53:58.864725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:58.864725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:58.864725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.864725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:58.864725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:58.864725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:53:58.864725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.865045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:53:58.865045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:58.865045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:53:58.865045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:58.865045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:53:58.865045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:53:58.865045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:58.865045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:53:58.865045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.865045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:53:58.865045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:53:58.865045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.865045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:53:58.865308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.865308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:53:58.865678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:53:58.865984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:53:58.866359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:53:58.866359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:53:58.866359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:01.147793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:01.148548 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:01.148548 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:01.148850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:01.183646 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:01.183818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:01.184427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:01.184427 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:01.184427 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:01.184427 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:01.184427 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:01.184427 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:01.184427 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:01.184427 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:01.185456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:01.185456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:01.185456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:01.185456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:01.185456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:01.185456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:01.185456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:01.185456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:01.185456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:01.185456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:01.185456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:01.185456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:01.186040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:01.186040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:01.186040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:01.186040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:01.186040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:01.186040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:01.186040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:01.186040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:01.186040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:01.186040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:01.186040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:01.186040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:01.186040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:01.186469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:01.186469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:01.186555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:01.187033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:01.187537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:01.187537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:01.187537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:03.840393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:03.841716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:03.841851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:03.842745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:03.843184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:03.843639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.844551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:03.844551 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.844551 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:03.844551 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:03.844551 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.844551 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:03.844551 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.844551 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:03.846128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.846128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:03.846128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:03.846128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:03.846128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:03.846128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:03.846128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:03.846128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.846128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:03.846128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:03.846128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:03.846128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.847001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:03.847001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:03.847001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:03.847001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:03.847001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:03.847001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:03.847001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:03.847001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:03.847001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.847001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:03.847001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:03.847001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.847001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:03.847649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.847649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:03.848469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:03.848932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.849445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:03.849445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.849445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:03.849934 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:03.850427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:03.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:03.850899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:54:03.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:03.851654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.852412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:03.852412 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.852412 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:03.852412 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:03.852412 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.852412 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:03.852412 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.852412 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:03.853747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:03.854248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:03.854248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:03.854248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.854248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:03.854248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:03.854248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.854248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:03.854248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.854248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:03.855286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.855286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:03.855286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:03.855286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:03.855286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:03.855286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:03.855286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:03.855286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:54:03.855286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:03.855286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:03.855286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.855286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:03.855672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:03.856035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.856035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:03.856035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.856035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:03.857352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:03.857813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:03.858311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.858311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:03.858311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:03.858311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.858311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:03.858311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.858311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:03.858311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:03.858311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:03.858311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:03.858941 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:03.858941 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:03.859791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.859904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:03.859904 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.859904 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:03.859904 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:03.859904 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.859904 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:03.859904 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.859904 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:03.860649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:03.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:03.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:03.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:03.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:03.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:03.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:03.861786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.861786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:03.861786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:03.861786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:03.861786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:03.861786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:03.861786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:03.861786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:03.861786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:03.861786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.861786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:03.862595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:03.863410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:03.864005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:03.864916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:03.864916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:03.864916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:03.864916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:03.864916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.864916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:03.864916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:03.864916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.864916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:03.864916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.864916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:03.864916 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:03.864916 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:03.865912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:03.865912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:03.865912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:03.865912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:03.865912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.865912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:03.865912 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.865912 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:03.865912 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:03.866561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:03.867354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:03.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:03.869105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:03.870535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:03.870535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:03.870535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:03.870535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:03.870535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:03.870535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:06.148512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:06.149952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:06.150335 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:06.151306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:06.184358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:06.184597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:06.185285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:06.185285 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:06.185285 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:06.185285 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:06.185285 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:06.185285 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:06.185285 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:06.185285 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:06.186293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:06.186293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:06.186293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:06.186293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:06.186293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:06.186293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:06.186293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:06.186293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:06.186293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:06.186293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:06.186293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:06.186293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:06.186875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:06.186875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:06.186875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:06.186875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:06.186875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:06.186875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:06.186875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:06.186875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:06.186875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:06.186875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:06.186875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:06.186875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:06.186875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:06.187406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:06.188020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:06.190048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:06.190048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:06.190048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:06.190048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:06.190048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:06.190048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:08.841788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:08.842482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:08.842727 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:08.843496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:08.843712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:08.844232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.844764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:08.844764 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.844764 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:08.844764 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:08.844764 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.844764 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:08.844764 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.844764 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:08.846991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.846991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:08.846991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:08.846991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:08.846991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:08.846991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:08.846991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:08.846991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.846991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:08.846991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:08.846991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:08.846991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.847779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:08.847779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:08.847779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:08.847779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:08.847779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:08.847779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:08.847779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:08.847779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:08.847779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.847779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:08.847779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:08.847779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.847779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:08.848305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:08.848818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.849419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:08.849419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:08.849419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.849419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:08.849419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.849419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:08.850450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:08.850859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:08.851034 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:08.851377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:54:08.852339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:08.852622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.853363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:08.853363 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.853363 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:08.853363 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:08.853363 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.853363 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:08.853363 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.853363 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:08.854189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:08.854679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:08.854679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:08.854679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.854679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:08.854679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:08.854679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.854679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:08.854679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.854679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:08.855158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.855158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:08.855158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:08.855158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:08.855158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:08.855158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:08.855158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:08.855158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:54:08.855158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:08.855158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:08.855158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.855158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:08.855534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:08.855911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.855911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:08.855911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.855911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:08.856508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:08.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:08.857490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.857490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:08.857490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:08.857490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.857490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:08.857490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.857490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:08.858148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:08.858612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:08.858749 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:08.858894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:08.859548 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:08.859804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.860330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:08.860330 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.860330 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:08.860330 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:08.860330 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.860330 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:08.860330 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.860330 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:08.861111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:08.861859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:08.861859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:08.861859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.861859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:08.861859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:08.861859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.861859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:08.861859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.861859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:08.862267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.862267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:08.862267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:08.862267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:08.862267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:08.862267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:08.862267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:08.862267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:08.862267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:08.862267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.862267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:08.862781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:08.863551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.863551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:08.863551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.863551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:08.863832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:08.864667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:08.865476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.865476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:08.865476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.865476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:08.865476 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:08.865476 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:08.865476 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:08.865476 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:08.865883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:08.865883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:08.865883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.866557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:08.866557 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.866557 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:08.866557 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:08.866557 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.866557 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:08.866557 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.866557 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:08.867798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.867798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:08.867798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:08.867798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:08.867798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:08.867798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:08.867798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:08.867798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.867798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:08.867798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:08.867798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:08.867798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.868489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:08.868489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:08.868489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:08.868489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:08.868489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:08.868489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:08.868489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:08.868489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:08.868489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.868489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:08.868489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:08.868489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.868489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:08.869253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.869253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:08.869997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:08.870764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:08.871744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:08.871744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:08.871744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:11.150039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:11.150838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:11.150838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:11.151948 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:11.153778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:54:11.182717 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:11.182969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:11.183490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:11.183490 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:11.183490 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:11.183490 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:11.183490 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:11.183490 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:11.183490 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:11.183490 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:11.184142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:11.184142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:11.184142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:11.184142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:11.184142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:11.184142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:11.184142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:11.184142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:11.184142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:11.184142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:11.184142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:11.184142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:11.184798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:11.184798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:11.184798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:11.184798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:11.184798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:11.184798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:11.184798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:11.184798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:11.184798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:11.184798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:11.184798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:11.184798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:11.184798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:11.185393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:11.186083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:11.186931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:11.186931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:11.186931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:11.186931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:11.186931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:11.186931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:13.842466 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:13.843702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:13.843702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:13.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:13.844079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:13.844240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.844931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:13.844931 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.844931 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:13.844931 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:13.844931 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.844931 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:13.844931 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.844931 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:13.845880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.845880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:13.845880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:13.845880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:13.845880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:13.845880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:13.845880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:13.845880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.845880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:13.845880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:13.845880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:13.845880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.846444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:13.846444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:13.846444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:13.846444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:13.846444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:13.846444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:13.846444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:13.846444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:13.846444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.846444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:13.846444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:13.846444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.846444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:13.847006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.847006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:13.847760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:13.848467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.849295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:13.849295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.849295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:13.850926 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:13.851257 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:13.851381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:13.851527 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:54:13.851908 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:13.852130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.852659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:13.852659 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.852659 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:13.852659 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:13.852659 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.852659 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:13.852659 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.852659 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:13.853450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:13.853869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:13.853869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:13.853869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.853869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:13.853869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:13.853869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.853869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:13.853869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.853869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:13.854369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.854369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:13.854369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:13.854369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:13.854369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:13.854369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:13.854369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:13.854369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:54:13.854369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:13.854369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:13.854369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.854369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:13.854766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:13.855124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.855124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:13.855124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.855124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:13.855600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:13.856080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:13.856797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.856797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:13.856797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:13.856797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.856797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:13.856797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.856797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:13.858657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:13.858955 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:13.859063 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:13.859195 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:13.859527 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:13.859779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.860328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:13.860328 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.860328 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:13.860328 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:13.860328 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.860328 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:13.860328 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.860328 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:13.861102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:13.861449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:13.861449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:13.861449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.861449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:13.861449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:13.861449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.861449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:13.861449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.861449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:13.861997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.861997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:13.861997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:13.861997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:13.861997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:13.861997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:13.861997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:13.861997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:13.861997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:13.861997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.861997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:13.862238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:13.862520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.862520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:13.862520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.862520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:13.863185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:13.863590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:13.864034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.864034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:13.864034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.864034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:13.864664 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:13.864976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:13.865089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:13.865264 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:13.865568 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:13.865803 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:13.865977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.866477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:13.866477 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.866477 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:13.866477 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:13.866477 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.866477 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:13.866477 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.866477 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:13.867346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.867346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:13.867346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:13.867346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:13.867346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:13.867346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:13.867346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:13.867346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.867346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:13.867346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:13.867346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:13.867346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.867647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:13.867647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:13.867647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:13.867647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:13.867647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:13.867647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:13.867647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:13.867647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:13.867647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.867647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:13.867647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:13.867647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.867647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:13.867896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.867896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:13.868716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:13.869041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:13.869420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:13.869420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:13.869420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:16.151013 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:16.152015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:16.152015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:16.153150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:16.188768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:16.189001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:16.189638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:16.189638 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:16.189638 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:16.189638 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:16.189638 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:16.189638 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:16.189638 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:16.189638 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:16.190594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:16.190594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:16.190594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:16.190594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:16.190594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:16.190594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:16.190594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:16.190594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:16.190594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:16.190594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:16.190594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:16.190594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:16.191175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:16.191175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:16.191175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:16.191175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:16.191175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:16.191175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:16.191175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:16.191175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:16.191175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:16.191175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:16.191175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:16.191175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:16.191175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:16.191725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:16.192353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:16.193084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:16.193084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:16.193084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:16.193084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:16.193084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:16.193084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:18.843092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:18.844015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:18.844015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:18.844597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:18.844810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:18.844921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.845532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:18.845532 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.845532 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:18.845532 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:18.845532 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.845532 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:18.845532 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.845532 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:18.847388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.847388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:18.847388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:18.847388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:18.847388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:18.847388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:18.847388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:18.847388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.847388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:18.847388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:18.847388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:18.847388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.848031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:18.848031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:18.848031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:18.848031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:18.848031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:18.848031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:18.848031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:18.848031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:18.848031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.848031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:18.848031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:18.848031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.848031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:18.848697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.848697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:18.849407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:18.850180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.851081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:18.851081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.851081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:18.851392 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:18.851689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:18.851792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:18.851907 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:54:18.852470 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:18.852678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.853436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:18.853436 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.853436 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:18.853436 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:18.853436 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.853436 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:18.853436 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.853436 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:18.854138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:18.854672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:18.854672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:18.854672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.854672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:18.854672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:18.854672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.854672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:18.854672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.854672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:18.854963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.854963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:18.854963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:18.854963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:18.854963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:18.854963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:18.854963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:18.854963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:54:18.854963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:18.854963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:18.854963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.854963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:18.855361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:18.855750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.855750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:18.855750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.855750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:18.855931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:18.856411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:18.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:18.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:18.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:18.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.856969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:18.859099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:18.859395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:18.859502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:18.859618 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:18.860096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:18.860487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.861226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:18.861226 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.861226 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:18.861226 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:18.861226 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.861226 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:18.861226 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.861226 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:18.861945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:18.862405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:18.862405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:18.862405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.862405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:18.862405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:18.862405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.862405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:18.862405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.862405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:18.862790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.862790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:18.862790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:18.862790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:18.862790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:18.862790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:18.862790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:18.862790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:18.862790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:18.862790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.862790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:18.863147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:18.863584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.863584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:18.863584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.863584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:18.863813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:18.864353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:18.864927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.864927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:18.864927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.864927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:18.865119 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:18.865460 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:18.865561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:18.865707 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:18.866301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:18.866824 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:18.867079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.867768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:18.867768 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.867768 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:18.867768 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:18.867768 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.867768 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:18.867768 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.867768 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:18.868601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.868601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:18.868601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:18.868601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:18.868601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:18.868601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:18.868601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:18.868601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.868601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:18.868601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:18.868601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:18.868601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.869076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:18.869076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:18.869076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:18.869076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:18.869076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:18.869076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:18.869076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:18.869076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:18.869076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.869076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:18.869076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:18.869076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.869076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:18.869478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.869478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:18.869599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:18.870085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:18.870650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:18.870650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:18.870650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:21.151485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:21.152350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:21.152350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:21.153665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:21.191876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:21.192268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:21.193116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:21.193116 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:21.193116 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:21.193116 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:21.193116 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:21.193116 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:21.193116 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:21.193116 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:21.194151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:21.194151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:21.194151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:21.194151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:21.194151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:21.194151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:21.194151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:21.194151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:21.194151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:21.194151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:21.194151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:21.194151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:21.195011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:21.195011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:21.195011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:21.195011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:21.195011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:21.195011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:21.195011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:21.195011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:21.195011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:21.195011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:21.195011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:21.195011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:21.195011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:21.195829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:21.196815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:21.197952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:21.197952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:21.197952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:21.197952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:21.197952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:21.197952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:23.843575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:23.849716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:23.849716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:23.849716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:23.849716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:23.849716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.849716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:23.849716 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.849716 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:23.849716 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:23.850882 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.850882 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:23.850882 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.850882 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:23.850882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.850882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:23.850882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:23.850882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:23.850882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:23.850882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:23.850882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:23.850882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.850882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:23.852080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:23.852895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:23.853663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:23.854689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:23.854689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:23.854689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.854689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:23.854689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:23.854689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.854689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:23.854689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.854689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:23.854689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:23.854689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:23.854689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:23.855320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:54:23.855320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:23.855320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.855320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:23.855320 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.855320 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:23.855320 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:23.855320 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.855320 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:23.855320 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.855320 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:23.855320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.855320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.855912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:23.856708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:23.856708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.856708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:23.856708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.856708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:23.856708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.856708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:23.856708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:23.856708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:23.856708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:23.856708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:23.856708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:23.856708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:54:23.857400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:23.857400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:23.857400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.857400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:23.857400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:23.857400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:23.857400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.857400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:23.857400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:23.857400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:23.857400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:23.857400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:23.857400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:23.857964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:23.857964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:23.857964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.857964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:23.857964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:23.857964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.857964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:23.857964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.857964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:23.857964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.857964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:23.857964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.858553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.859185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:23.859695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:23.859866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:23.859973 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:23.860097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:23.860475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:23.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.861318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:23.861318 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.861318 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:23.861318 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:23.861318 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.861318 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:23.861318 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.861318 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:23.862049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:23.862486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:23.862486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:23.862486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.862486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:23.862486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:23.862486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.862486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:23.862486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.862486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:23.862929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.862929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:23.862929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:23.862929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:23.862929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:23.862929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:23.862929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:23.862929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:23.862929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:23.862929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.862929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:23.863272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:23.863622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.863622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:23.863622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.863622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:23.863961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:23.864484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:23.865014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.865014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:23.865014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.865014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:23.865636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:23.865922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:23.866032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:23.866202 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:23.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:23.866856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:23.867001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.867556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:23.867556 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.867556 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:23.867556 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:23.867556 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.867556 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:23.867556 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.867556 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:23.868939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.868939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:23.868939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:23.868939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:23.868939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:23.868939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:23.868939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:23.868939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.868939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:23.868939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:23.868939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:23.868939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.869648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:23.869648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:23.869648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:23.869648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:23.869648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:23.869648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:23.869648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:23.869648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:23.869648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.869648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:23.869648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:23.869648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.869648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:23.870391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:23.871098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:23.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:23.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:23.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:23.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:23.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:23.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:26.152145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:26.153043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:26.153043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:26.153609 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:26.183919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:26.184158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:26.185570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:26.185570 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:26.185570 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:26.185570 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:26.185570 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:26.185570 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:26.185570 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:26.185570 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:26.186606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:26.186606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:26.186606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:26.186606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:26.186606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:26.186606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:26.186606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:26.186606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:26.186606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:26.186606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:26.186606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:26.186606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:26.186983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:26.186983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:26.186983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:26.186983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:26.186983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:26.186983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:26.186983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:26.186983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:26.186983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:26.186983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:26.186983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:26.186983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:26.186983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:26.187387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:26.187387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:26.187704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:26.188159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:26.188718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:26.188718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:26.188718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:28.845271 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:28.845271 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:28.845271 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:28.846387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:28.846387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:28.846387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.846587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:28.846587 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.846587 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:28.846587 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:28.846587 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.846587 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:28.846587 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.846587 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:28.848184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.848184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:28.848184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:28.848184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:28.848184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:28.848184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:28.848184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:28.848184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.848184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:28.848184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:28.848184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:28.848184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.848777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:28.848777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:28.848777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:28.848777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:28.848777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:28.848777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:28.848777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:28.848777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:28.848777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.848777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:28.848777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:28.848777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.848777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:28.849125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.849125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:28.850016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:28.850653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.851179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:28.851179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.851179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:28.852132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:28.852561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:28.852603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:28.852747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:54:28.852984 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:28.853127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.853531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:28.853531 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.853531 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:28.853531 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:28.853531 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.853531 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:28.853531 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.853531 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:28.854304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:28.854905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:28.854905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:28.854905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.854905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:28.854905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:28.854905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.854905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:28.854905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.854905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:28.855300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.855300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:28.855300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:28.855300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:28.855300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:28.855300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:28.855300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:28.855300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:54:28.855300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:28.855300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:28.855300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.855300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:28.855981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:28.856548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:28.857135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:28.857979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:28.857979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:28.857979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:28.857979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:28.857979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:28.857979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.857979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:28.857979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:28.857979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.857979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:28.857979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.857979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:28.860073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:28.860363 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:28.860472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:28.860593 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:28.861001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:28.861163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.861557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:28.861557 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.861557 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:28.861557 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:28.861557 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.861557 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:28.861557 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.861557 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:28.862418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:28.862835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:28.862835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:28.862835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.862835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:28.862835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:28.862835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.862835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:28.862835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.862835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:28.863261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.863261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:28.863261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:28.863261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:28.863261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:28.863261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:28.863261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:28.863261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:28.863261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:28.863261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.863261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:28.863585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:28.863963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.863963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:28.863963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.863963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:28.864277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:28.864764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:28.865288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.865288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:28.865288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.865288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:28.865996 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:28.866203 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:28.866326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:28.866466 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:28.866713 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:28.866953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:28.866953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.867557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:28.867557 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.867557 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:28.867557 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:28.867557 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.867557 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:28.867557 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.867557 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:28.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:28.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:28.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:28.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:28.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:28.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:28.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:28.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:28.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:28.868229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.868779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:28.868779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:28.868779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:28.868779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:28.868779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:28.868779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:28.868779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:28.868779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:28.868779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.868779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:28.868779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:28.868779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.868779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:28.869315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:28.870133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:28.870872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:28.870872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:28.870872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:28.870872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:28.870872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:28.870872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:31.152723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:31.153523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:31.153523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:31.154248 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:31.210893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:31.211223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:31.212182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:31.212182 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:31.212182 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:31.212182 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:31.212182 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:31.212182 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:31.212182 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:31.212182 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:31.213358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.059 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:31.213358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:31.213358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:31.213358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:31.213358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:31.213358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:31.213358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:31.213358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:31.213358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:31.213358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:31.213358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:31.213358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:31.214461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:31.214461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:31.214461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:31.214461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:31.214461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:31.214461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:31.214461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:31.214461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:31.214461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:31.214461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:31.214461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:31.214461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:31.214461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:31.214777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:31.215158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:31.215610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:31.215610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:31.215610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:31.215610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:31.215610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:31.215610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:33.844831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:33.845738 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:33.845851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:33.847124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:33.847477 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:33.847693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.848742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:33.848742 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.848742 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:33.848742 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:33.848742 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.848742 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:33.848742 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.848742 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:33.850010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.850010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:33.850010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:33.850010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:33.850010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:33.850010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:33.850010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:33.850010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.850010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:33.850010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:33.850010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:33.850010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.850713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:33.850713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:33.850713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:33.850713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:33.850713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:33.850713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:33.850713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:33.850713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:33.850713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.850713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:33.850713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:33.850713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.850713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:33.851335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:33.852062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.852974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:33.852974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:33.852974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.852974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:33.852974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.852974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:33.852974 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:33.852974 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:33.853427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:33.853427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:54:33.853834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:33.854126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.855259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:33.855259 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.855259 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:33.855259 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:33.855259 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.855259 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:33.855259 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.855259 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:33.856441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:33.856892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:33.856892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:33.856892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.856892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:33.856892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:33.856892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.856892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:33.856892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.856892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:33.857453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.857453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:33.857453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:33.857453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:33.857453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:33.857453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:33.857453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:33.857453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:54:33.857453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:33.857453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:33.857453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.857453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:33.857797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:33.858168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.858168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:33.858168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.858168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:33.858665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:33.859101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:33.859620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.859620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:33.859620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:33.859620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.859620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:33.859620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.859620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:33.860641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:33.861518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:33.861629 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:33.861786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:33.862569 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:33.862952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.864104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:33.864104 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.864104 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:33.864104 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:33.864104 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.864104 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:33.864104 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.864104 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:33.865200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:33.865615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:33.865615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:33.865615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.865615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:33.865615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:33.865615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.865615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:33.865615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.865615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:33.866166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.866166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:33.866166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:33.866166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:33.866166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:33.866166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:33.866166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:33.866166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:33.866166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:33.866166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.866166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:33.866513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:33.866894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.866894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:33.866894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.866894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:33.867700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:33.868187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:33.868677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.868677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:33.868677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.868677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:33.868677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:33.868677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:33.868677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:33.868895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:33.869371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:33.869708 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:33.869926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.871057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:33.871057 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.871057 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:33.871057 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:33.871057 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.871057 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:33.871057 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.871057 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:33.872394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.872394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:33.872394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:33.872394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:33.872394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:33.872394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:33.872394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:33.872394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.872394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:33.872394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:33.872394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:33.872394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.872743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:33.872743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:33.872743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:33.872743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:33.872743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:33.872743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:33.872743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:33.872743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:33.872743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.872743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:33.872743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:33.872743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.872743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:33.873091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.873091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:33.873622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:33.874063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:33.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:33.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:33.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:36.153416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:36.154063 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:36.154318 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:36.155526 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:36.188730 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:36.189114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:36.189975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:36.189975 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:36.189975 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:36.189975 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:36.189975 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:36.189975 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:36.189975 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:36.189975 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:36.190980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:36.190980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:36.190980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:36.190980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:36.190980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:36.190980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:36.190980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:36.190980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:36.190980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:36.190980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:36.190980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:36.190980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:36.191628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:36.191628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:36.191628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:36.191628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:36.191628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:36.191628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:36.191628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:36.191628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:36.191628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:36.191628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:36.191628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:36.191628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:36.191628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:36.192238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:36.192938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:36.193762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:36.193762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:36.193762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:36.193762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:36.193762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:36.193762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:38.845872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:38.847007 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:38.847007 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:38.847561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:38.847979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:38.848267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.849017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:38.849017 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.849017 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:38.849017 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:38.849017 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.849017 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:38.849017 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.849017 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:38.850045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.850045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:38.850045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:38.850045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:38.850045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:38.850045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:38.850045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:38.850045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.850045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:38.850045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:38.850045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:38.850045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:38.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:38.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:38.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:38.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:38.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:38.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:38.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:38.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:38.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:38.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.850678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:38.851355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:38.852128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.853161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:38.853161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:38.853161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.853161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:38.853161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.853161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:38.853161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:38.853554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:38.853554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:38.853554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:54:38.853927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:38.854083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.854668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:38.854668 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.854668 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:38.854668 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:38.854668 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.854668 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:38.854668 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.854668 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:38.855966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:38.856638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:38.856638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:38.856638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.856638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:38.856638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:38.856638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.856638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:38.856638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.856638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:38.857450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.857450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:38.857450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:38.857450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:38.857450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:38.857450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:38.857450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:38.857450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:54:38.857450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:38.857450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:38.857450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.857450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:38.857846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:38.858498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.858498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:38.858498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.858498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:38.859132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:38.859597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:38.860129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.860129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:38.860129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:38.860129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.860129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:38.860129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.860129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:38.861054 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:38.861551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:38.861748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:38.861930 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:38.862433 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:38.862643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.863152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:38.863152 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.863152 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:38.863152 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:38.863152 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.863152 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:38.863152 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.863152 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:38.863930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:38.864371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:38.864371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:38.864371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.864371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:38.864371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:38.864371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.864371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:38.864371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.864371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:38.864789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.864789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:38.864789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:38.864789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:38.864789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:38.864789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:38.864789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:38.864789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:38.864789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:38.864789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.864789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:38.865138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:38.865558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.865558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:38.865558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.865558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:38.865811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:38.866335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:38.866845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.866845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:38.866845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.866845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:38.868426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:38.868604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:38.868701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:38.868850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:38.869150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:38.869351 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:38.869457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.869885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:38.869885 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.869885 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:38.869885 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:38.869885 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.869885 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:38.869885 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.869885 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:38.870679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.870679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:38.870679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:38.870679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:38.870679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:38.870679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:38.870679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:38.870679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.870679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:38.870679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:38.870679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:38.870679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.871051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:38.871051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:38.871051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:38.871051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:38.871051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:38.871051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:38.871051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:38.871051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:38.871051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.871051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:38.871051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:38.871051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.871051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:38.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:38.871619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:38.872163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:38.872677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:38.872677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:38.872677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:41.154103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:41.154721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:41.154790 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:41.156033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:41.198766 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:41.199158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:41.200044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:41.200044 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:41.200044 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:41.200044 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:41.200044 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:41.200044 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:41.200044 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:41.200044 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:41.200875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:41.200875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:41.200875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:41.200875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:41.200875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:41.200875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:41.200875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:41.200875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:41.200875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:41.200875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:41.200875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:41.200875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:41.201699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:41.201699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:41.201699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:41.201699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:41.201699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:41.201699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:41.201699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:41.201699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:41.201699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:41.201699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:41.201699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:41.201699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:41.201699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:41.202472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:41.203287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:41.204033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:41.204033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:41.204033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:41.204033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:41.204033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:41.204033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:43.846550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:43.847291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:43.847291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:43.847865 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:43.848416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:43.848687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.849462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:43.849462 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.849462 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:43.849462 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:43.849462 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.849462 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:43.849462 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.849462 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:43.850232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.850232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:43.850232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:43.850232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:43.850232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:43.850232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:43.850232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:43.850232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.850232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:43.850232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:43.850232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:43.850232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.850916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:43.850916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:43.850916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:43.850916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:43.850916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:43.850916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:43.850916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:43.850916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:43.850916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.850916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:43.850916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:43.850916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.850916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:43.851641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:43.852521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.853595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:43.853595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:43.853595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.853595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:43.853595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.853595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:43.853595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:43.853595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:43.853595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:43.854112 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:54:43.854112 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:43.854501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.855166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:43.855166 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.855166 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:43.855166 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:43.855166 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.855166 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:43.855166 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.855166 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:43.855728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:43.856222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:43.856222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:43.856222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.856222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:43.856222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:43.856222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.856222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:43.856222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.856222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:43.856518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.856518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:43.856518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:43.856518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:43.856518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:43.856518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:43.856518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:43.856518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:54:43.856518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:43.856518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:43.856518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.856518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:43.856875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:43.857267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:43.857655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:43.858172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:43.858172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:43.858172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:43.858172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:43.858172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:43.858172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.858172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:43.858172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:43.858172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.858172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:43.858172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.858172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:43.861603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:43.861859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:43.861937 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:43.862030 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:43.862414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:43.862635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.863115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:43.863115 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.863115 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:43.863115 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:43.863115 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.863115 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:43.863115 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.863115 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:43.863653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:43.864063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:43.864063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:43.864063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.864063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:43.864063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:43.864063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.864063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:43.864063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.864063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:43.864338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.864338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:43.864338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:43.864338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:43.864338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:43.864338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:43.864338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:43.864338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:43.864338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:43.864338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.864338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:43.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:43.865088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:43.865527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:43.866105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:43.866105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:43.866105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:43.866105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:43.866105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.866105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:43.866105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:43.866105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.866105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:43.866105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.866105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:43.868791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:43.869009 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:43.869065 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:43.869190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:43.869628 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:43.869974 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:43.870198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.870982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:43.870982 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.870982 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:43.870982 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:43.870982 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.870982 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:43.870982 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.870982 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:43.871598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.871598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:43.871598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:43.871598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:43.871598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:43.871598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:43.871598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:43.871598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.871598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:43.871598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:43.871598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:43.871598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.871949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:43.871949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:43.871949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:43.871949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:43.871949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:43.871949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:43.871949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:43.871949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:43.871949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.871949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:43.871949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:43.871949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.871949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:43.872315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:43.872768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:43.873302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:43.873302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:43.873302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:43.873302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:43.873302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:43.873302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:46.154726 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:46.155716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:46.155716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:46.156872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:46.188235 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:46.188530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:46.188937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:46.188937 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:46.188937 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:46.188937 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:46.188937 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:46.188937 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:46.188937 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:46.188937 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:46.189769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:46.189769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:46.189769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:46.189769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:46.189769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:46.189769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:46.189769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:46.189769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:46.189769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:46.189769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:46.189769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:46.189769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:46.190329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:46.190329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:46.190329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:46.190329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:46.190329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:46.190329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:46.190329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:46.190329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:46.190329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:46.190329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:46.190329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:46.190329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:46.190329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:46.190849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:46.191501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:46.192580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:46.192580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:46.192580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:46.192580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:46.192580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:46.192580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:48.847139 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:48.847972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:48.847972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:48.848903 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:48.849088 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:48.849230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.849808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:48.849808 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.849808 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:48.849808 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:48.849808 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.849808 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:48.849808 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.849808 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:48.851099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.851099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:48.851099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:48.851099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:48.851099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:48.851099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:48.851099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:48.851099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.851099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:48.851099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:48.851099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:48.851099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.851745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:48.851745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:48.851745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:48.851745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:48.851745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:48.851745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:48.851745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:48.851745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:48.851745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.851745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:48.851745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:48.851745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.851745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:48.852422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.852422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:48.853940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:48.854532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.855145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:48.855145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.855145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:48.855145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:48.855145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:48.855145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:48.855368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:54:48.855756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:48.855896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.856355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:48.856355 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.856355 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:48.856355 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:48.856355 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.856355 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:48.856355 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.856355 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:48.857313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:48.857824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:48.857824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:48.857824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.857824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:48.857824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:48.857824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.857824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:48.857824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.857824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:48.858227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.858227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:48.858227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:48.858227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:48.858227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:48.858227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:48.858227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:48.858227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:54:48.858227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:48.858227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:48.858227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.858227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:48.858636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:48.859073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.859073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:48.859073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.859073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:48.859447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:48.859933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:48.860503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.860503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:48.860503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:48.860503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.860503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:48.860503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.860503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:48.861918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:48.862218 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:48.862359 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:48.862486 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:48.862752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:48.862914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.863336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:48.863336 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.863336 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:48.863336 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:48.863336 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.863336 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:48.863336 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.863336 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:48.864089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:48.864621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:48.864621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:48.864621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.864621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:48.864621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:48.864621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.864621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:48.864621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.864621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:48.864996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.864996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:48.864996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:48.864996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:48.864996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:48.864996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:48.864996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:48.864996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:48.864996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:48.864996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.864996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:48.865466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:48.865852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.865852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:48.865852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.865852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:48.866030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:48.866541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:48.867064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.867064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:48.867064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.867064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:48.869139 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:48.869396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:48.869507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:48.869657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:48.869901 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:48.870062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:48.870159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.870615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:48.870615 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.870615 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:48.870615 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:48.870615 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.870615 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:48.870615 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.870615 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:48.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:48.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:48.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:48.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:48.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:48.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:48.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:48.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:48.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:48.871522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.871946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:48.871946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:48.871946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:48.871946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:48.871946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:48.871946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:48.871946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:48.871946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:48.871946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.871946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:48.871946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:48.871946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.871946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:48.872357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.872357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:48.872779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:48.873265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:48.873801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:48.873801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:48.873801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:51.155273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:51.155792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:51.155792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:51.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:51.188873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:51.189335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:51.190271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:51.190271 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:51.190271 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:51.190271 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:51.190271 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:51.190271 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:51.190271 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:51.190271 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:51.191368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:51.191368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:51.191368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:51.191368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:51.191368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:51.191368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:51.191368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:51.191368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:51.191368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:51.191368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:51.191368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:51.191368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:51.191950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:51.191950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:51.191950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:51.191950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:51.191950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:51.191950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:51.191950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:51.191950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:51.191950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:51.191950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:51.191950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:51.191950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:51.191950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:51.192652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:51.193390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:51.194265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:51.194265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:51.194265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:51.194265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:51.194265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:51.194265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:53.847869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:53.848623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:53.848623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:53.849666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:53.849998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:53.850257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.851108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:53.851108 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.851108 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:53.851108 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:53.851108 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.851108 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:53.851108 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.851108 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:53.852787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.852787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:53.852787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:53.852787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:53.852787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:53.852787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:53.852787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:53.852787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.852787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:53.852787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:53.852787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:53.852787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.853370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:53.853370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:53.853370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:53.853370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:53.853370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:53.853370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:53.853370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:53.853370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:53.853370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.853370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:53.853370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:53.853370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.853370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:53.853986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:53.854681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.855610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:53.855610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:53.855610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.855610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:53.855610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.855610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:53.855610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:53.855610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:53.855610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:53.855610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:54:53.856100 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:53.856166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.856940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:53.856940 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.856940 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:53.856940 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:53.856940 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.856940 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:53.856940 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.856940 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:53.857685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:53.858423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:53.858423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:53.858423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.858423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:53.858423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:53.858423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.858423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:53.858423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.858423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:53.858836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.858836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:53.858836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:53.858836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:53.858836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:53.858836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:53.858836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:53.858836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:54:53.858836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:53.858836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:53.858836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.858836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:53.859488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:53.860289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:53.860990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:53.861920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:53.861920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:53.861920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:53.861920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:53.861920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:53.861920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.861920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:53.861920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:53.861920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.861920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:53.861920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.861920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:53.862492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:53.862586 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:53.862717 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:53.862812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:53.863309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:53.863648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.864294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:53.864294 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.864294 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:53.864294 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:53.864294 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.864294 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:53.864294 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.864294 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:53.865044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:53.865343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:53.865343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:53.865343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.865343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:53.865343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:53.865343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.865343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:53.865343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.865343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:53.865939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.865939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:53.865939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:53.865939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:53.865939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:53.865939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:53.865939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:53.865939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:53.865939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:53.865939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.865939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:53.866173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:53.866448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.866448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:53.866448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.866448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:53.867038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:53.867362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:53.867720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.867720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:53.867720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.867720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:53.869501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:53.869756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:53.869864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:53.870011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:53.870502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:53.870843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:53.871060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.871753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:53.871753 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.871753 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:53.871753 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:53.871753 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.871753 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:53.871753 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.871753 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:53.872651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.872651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:53.872651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:53.872651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:53.872651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:53.872651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:53.872651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:53.872651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.872651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:53.872651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:53.872651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:53.872651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:53.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:53.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:53.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:53.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:53.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:53.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:53.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:53.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:53.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:53.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:53.873850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:53.874601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:53.875539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:53.875539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:53.875539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:53.875539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:53.875539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:53.875539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:56.155891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:56.156768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:56.156768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:56.157978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:56.191173 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:56.191624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:56.192489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:56.192489 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:56.192489 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:56.192489 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:56.192489 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:56.192489 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:56.192489 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:56.192489 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:56.193463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:56.193463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:56.193463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:56.193463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:56.193463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:56.193463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:56.193463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:56.193463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:56.193463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:56.193463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:56.193463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:56.193463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:56.194012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:56.194012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:56.194012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:56.194012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:56.194012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:56.194012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:56.194012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:56.194012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:56.194012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:56.194012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:56.194012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:56.194012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:56.194012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:56.195133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:56.196367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:56.197109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:56.197109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:56.197109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:56.197109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:56.197109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:56.197109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:58.848525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:58.849514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:58.850599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:58.850959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:58.851252 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:58.851400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:58.856870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:58.857200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.857494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:58.857494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:58.857494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.857494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:58.857494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.857494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:58.858789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:58.859111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.859475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:58.859475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.859475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:58.860179 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:58.860885 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:58.860885 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:58.860885 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:54:58.861489 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:58.861706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.863592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.863967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:58.865501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.865501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:58.865501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:58.865501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:58.865501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:58.865501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:58.865501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:58.865501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:54:58.865501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:58.865501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:58.865501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.865501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:58.868151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:58.868483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.868483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:58.868483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.868483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:58.868483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:58.868483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.868483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:58.868483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:58.868483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:58.868483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:58.868483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:58.868483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:58.868819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:58.869368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:58.869701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:58.869701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:58.869701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:58.869701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:58.874098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.874430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:54:58.876760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.876760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:58.876760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:58.876760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:58.876760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:58.876760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:58.876760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:58.876760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:58.876760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:58.876760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.876760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:58.877374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:58.877964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:58.878411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:58.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:58.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:58.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:58.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:58.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:58.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:58.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:58.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:54:58.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:54:58.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:54:58.879010 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:54:58.879010 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:58.879010 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:54:58.879010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:58.880419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:54:58.881304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.882046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:54:58.882046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:54:58.882046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.882046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:54:58.882046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.882046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:54:58.882407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:54:58.883280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:54:58.884167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:54:58.884167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:54:58.884167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:01.156507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:01.157281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:01.157281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:01.157870 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:01.190311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:01.190543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:01.191459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:01.191459 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:01.191459 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:01.191459 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:01.191459 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:01.191459 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:01.191459 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:01.191459 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:01.192513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:01.192513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:01.192513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:01.192513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:01.192513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:01.192513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:01.192513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:01.192513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:01.192513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:01.192513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:01.192513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:01.192513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:01.193201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:01.193201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:01.193201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:01.193201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:01.193201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:01.193201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:01.193201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:01.193201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:01.193201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:01.193201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:01.193201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:01.193201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:01.193201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:01.193877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:01.194645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:01.195573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:01.195573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:01.195573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:01.195573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:01.195573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:01.195573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:03.849309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:03.850438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:03.850438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:03.851389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:03.851673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:03.851863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.852590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:03.852590 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.852590 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:03.852590 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:03.852590 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.852590 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:03.852590 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.852590 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:03.854350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.854350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:03.854350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:03.854350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:03.854350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:03.854350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:03.854350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:03.854350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.854350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:03.854350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:03.854350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:03.854350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.855149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:03.855149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:03.855149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:03.855149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:03.855149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:03.855149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:03.855149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:03.855149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:03.855149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.855149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:03.855149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:03.855149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.855149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:03.856639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.856639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:03.856807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:03.857606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.858126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:03.858126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.858126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:03.860732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:03.861044 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:03.861190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:03.861381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:55:03.861793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:03.862012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.862534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:03.862534 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.862534 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:03.862534 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:03.862534 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.862534 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:03.862534 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.862534 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:03.863412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:03.864124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:03.864124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:03.864124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.864124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:03.864124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:03.864124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.864124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:03.864124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.864124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:03.864636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.864636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:03.864636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:03.864636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:03.864636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:03.864636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:03.864636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:03.864636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:55:03.864636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:03.864636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:03.864636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.864636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:03.865147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:03.865758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.865758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:03.865758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.865758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:03.866284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:03.866794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:03.867329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.867329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:03.867329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:03.867329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.867329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:03.867329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.867329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:03.868003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:03.868341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:03.868486 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:03.868665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:03.869073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:03.869379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.869927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:03.869927 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.869927 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:03.869927 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:03.869927 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.869927 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:03.869927 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.869927 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:03.870805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:03.871394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:03.871394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:03.871394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.871394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:03.871394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:03.871394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.871394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:03.871394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.871394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:03.872028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.872028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:03.872028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:03.872028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:03.872028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:03.872028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:03.872028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:03.872028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:03.872028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:03.872028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.872028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:03.872547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:03.873099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.873099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:03.873099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.873099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:03.873530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:03.874053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:03.874584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.874584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:03.874584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.874584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:03.876777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:03.877088 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:03.877256 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:03.877459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:03.878061 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:03.878349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:03.878513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.879112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:03.879112 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.879112 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:03.879112 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:03.879112 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.879112 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:03.879112 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.879112 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:03.880102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.880102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:03.880102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:03.880102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:03.880102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:03.880102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:03.880102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:03.880102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.880102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:03.880102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:03.880102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:03.880102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:03.881241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.881241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:03.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:03.882067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:03.882588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:03.882588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:03.882588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:06.157090 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:06.157703 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:06.157876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:06.159142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:06.195823 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:06.196112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:06.196780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:06.196780 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:06.196780 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:06.196780 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:06.196780 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:06.196780 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:06.196780 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:06.196780 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:06.197569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:06.197569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:06.197569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:06.197569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:06.197569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:06.197569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:06.197569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:06.197569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:06.197569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:06.197569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:06.197569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:06.197569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:06.198192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:06.198192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:06.198192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:06.198192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:06.198192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:06.198192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:06.198192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:06.198192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:06.198192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:06.198192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:06.198192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:06.198192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:06.198192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:06.199050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:06.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:06.200631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:06.200631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:06.200631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:06.200631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:06.200631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:06.200631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:08.850062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:08.852383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:08.852383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:08.853673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:08.853855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:08.854110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.855226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:08.855226 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.855226 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:08.855226 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:08.855226 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.855226 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:08.855226 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.855226 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:08.856906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.856906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:08.856906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:08.856906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:08.856906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:08.856906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:08.856906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:08.856906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.856906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:08.856906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:08.856906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:08.856906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.857528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:08.857528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:08.857528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:08.857528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:08.857528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:08.857528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:08.857528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:08.857528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:08.857528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.857528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:08.857528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:08.857528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.857528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:08.858089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.858089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:08.858697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:08.859491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.860132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:08.860132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.860132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:08.861365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:08.861583 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:08.861680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:08.861815 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:55:08.862245 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:08.862476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.863092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:08.863092 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.863092 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:08.863092 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:08.863092 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.863092 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:08.863092 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.863092 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:08.864091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:08.864661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:08.864661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:08.864661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.864661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:08.864661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:08.864661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.864661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:08.864661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.864661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:08.864964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.864964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:08.864964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:08.864964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:08.864964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:08.864964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:08.864964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:08.864964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:55:08.864964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:08.864964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:08.864964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.864964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:08.865401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:08.865797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.865797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:08.865797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.865797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:08.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:08.866392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:08.866945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.866945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:08.866945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:08.866945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.866945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:08.866945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.866945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:08.868472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:08.868766 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:08.868952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:08.869036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:08.869506 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:08.869756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.870488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:08.870488 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.870488 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:08.870488 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:08.870488 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.870488 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:08.870488 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.870488 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:08.871352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:08.872495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:08.872773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:08.872773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:08.872773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:08.872773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:08.872773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.872773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:08.872773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:08.872773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:08.872773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.872773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:08.872773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:08.872773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:08.872773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:08.873015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:08.873015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:08.873015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:08.873015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:08.873015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.873015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:08.873015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:08.873015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.873015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:08.873015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.873015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:08.876762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:08.878348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:08.878715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.878715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:08.878715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.878715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:08.878715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:08.878715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:08.878715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:08.878715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:08.878920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:08.878920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:08.878920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.879174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:08.879174 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.879174 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:08.879174 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:08.879174 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.879174 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:08.879174 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.879174 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:08.880047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.880047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:08.880047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:08.880047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:08.880047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:08.880047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:08.880047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:08.880047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.880047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:08.880047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:08.880047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:08.880047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.880346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:08.880346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:08.880346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:08.880346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:08.880346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:08.880346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:08.880346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:08.880346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:08.880346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.880346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:08.880346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:08.880346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.880346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:08.880611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.880611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:08.881013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:08.881341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:08.881706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:08.881706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:08.881706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:11.157787 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:11.158703 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:11.158703 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:11.159993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:11.161878 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:55:11.194254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:11.194777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:11.195674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:11.195674 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:11.195674 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:11.195674 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:11.195674 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:11.195674 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:11.195674 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:11.195674 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:11.196513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:11.196513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:11.196513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:11.196513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:11.196513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:11.196513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:11.196513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:11.196513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:11.196513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:11.196513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:11.196513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:11.196513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:11.197404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:11.197404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:11.197404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:11.197404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:11.197404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:11.197404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:11.197404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:11.197404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:11.197404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:11.197404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:11.197404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:11.197404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:11.197404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:11.198337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:11.199296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:11.200312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:11.200312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:11.200312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:11.200312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:11.200312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:11.200312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:13.851539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:13.852500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:13.852500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:13.853396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:13.853639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:13.853797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.854486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:13.854486 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.854486 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:13.854486 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:13.854486 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.854486 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:13.854486 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.854486 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:13.856165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.856165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:13.856165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:13.856165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:13.856165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:13.856165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:13.856165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:13.856165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.856165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:13.856165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:13.856165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:13.856165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.856757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:13.856757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:13.856757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:13.856757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:13.856757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:13.856757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:13.856757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:13.856757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:13.856757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.856757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:13.856757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:13.856757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.856757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:13.857386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.857386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:13.858442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:13.859176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.860031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:13.860031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.860031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:13.861615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:13.862819 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:13.863058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:13.863356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:55:13.863866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:13.864119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.864879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:13.864879 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.864879 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:13.864879 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:13.864879 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.864879 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:13.864879 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.864879 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:13.866246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:13.866993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:13.866993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:13.866993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.866993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:13.866993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:13.866993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.866993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:13.866993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.866993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:13.867584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.867584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:13.867584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:13.867584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:13.867584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:13.867584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:13.867584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:13.867584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:55:13.867584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:13.867584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:13.867584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.867584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:13.868142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:13.868767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.868767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:13.868767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.868767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:13.869126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:13.869790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:13.870510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.870510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:13.870510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:13.870510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.870510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:13.870510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.870510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:13.870510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:13.870510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:13.870510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:13.870878 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:13.870878 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:13.870878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.871675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:13.871675 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.871675 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:13.871675 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:13.871675 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.871675 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:13.871675 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.871675 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:13.872592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:13.873188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:13.873188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:13.873188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.873188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:13.873188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:13.873188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.873188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:13.873188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.873188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:13.873820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.873820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:13.873820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:13.873820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:13.873820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:13.873820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:13.873820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:13.873820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:13.873820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:13.873820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.873820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:13.874359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:13.874968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.874968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:13.874968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.874968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:13.875341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:13.875720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:13.876163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.876163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:13.876163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.876163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:13.879450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:13.879667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:13.879779 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:13.879931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:13.880285 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:13.880507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:13.880653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.881154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:13.881154 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.881154 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:13.881154 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:13.881154 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.881154 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:13.881154 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.881154 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:13.881899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.881899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:13.881899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:13.881899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:13.881899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:13.881899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:13.881899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:13.881899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.881899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:13.881899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:13.881899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:13.881899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.882329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:13.882329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:13.882329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:13.882329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:13.882329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:13.882329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:13.882329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:13.882329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:13.882329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.882329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:13.882329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:13.882329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.882329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:13.882712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.882712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:13.882911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:13.883257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:13.883587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:13.883587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:13.883587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:16.158365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:16.159504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:16.159504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:16.160327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:16.193593 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:16.194037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:16.194928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:16.194928 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:16.194928 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:16.194928 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:16.194928 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:16.194928 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:16.194928 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:16.194928 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:16.195725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:16.195725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:16.195725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:16.195725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:16.195725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:16.195725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:16.195725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:16.195725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:16.195725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:16.195725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:16.195725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:16.195725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:16.196014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:16.196014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:16.196014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:16.196014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:16.196014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:16.196014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:16.196014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:16.196014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:16.196014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:16.196014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:16.196014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:16.196014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:16.196014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:16.196322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:16.196322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:16.196487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:16.196858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:16.197292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:16.197292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:16.197292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:18.852232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:18.853705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:18.853705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:18.854238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:18.854605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:18.854862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.855840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:18.855840 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.855840 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:18.855840 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:18.855840 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.855840 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:18.855840 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.855840 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:18.856830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.856830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:18.856830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:18.856830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:18.856830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:18.856830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:18.856830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:18.856830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.856830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:18.856830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:18.856830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:18.856830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.857089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:18.857089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:18.857089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:18.857089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:18.857089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:18.857089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:18.857089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:18.857089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:18.857089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.857089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:18.857089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:18.857089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.857089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:18.857334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.857334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:18.857883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:18.858548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.859317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:18.859317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.859317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:18.862727 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:18.862977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:18.863090 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:18.863236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:55:18.863555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:18.863749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.864240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:18.864240 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.864240 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:18.864240 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:18.864240 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.864240 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:18.864240 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.864240 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:18.864947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:18.865285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:18.865285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:18.865285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.865285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:18.865285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:18.865285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.865285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:18.865285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.865285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:18.865791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.865791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:18.865791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:18.865791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:18.865791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:18.865791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:18.865791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:18.865791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:55:18.865791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:18.865791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:18.865791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.865791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:18.866039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:18.866289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.866289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:18.866289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.866289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:18.866779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:18.867081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:18.867421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.867421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:18.867421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:18.867421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.867421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:18.867421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.867421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:18.869817 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:18.870103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:18.870232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:18.870354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:18.870794 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:18.871141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.871865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:18.871865 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.871865 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:18.871865 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:18.871865 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.871865 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:18.871865 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.871865 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:18.872756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:18.873047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:18.873047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:18.873047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.873047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:18.873047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:18.873047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.873047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:18.873047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.873047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:18.873607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.873607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:18.873607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:18.873607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:18.873607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:18.873607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:18.873607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:18.873607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:18.873607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:18.873607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.873607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:18.873836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:18.874076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.874076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:18.874076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.874076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:18.874632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:18.874952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:18.875286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.875286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:18.875286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.875286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:18.879821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:18.880116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:18.880250 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:18.880394 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:18.880822 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:18.881148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:18.881397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.882187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:18.882187 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.882187 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:18.882187 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:18.882187 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.882187 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:18.882187 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.882187 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:18.883771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.883771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:18.883771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:18.883771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:18.883771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:18.883771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:18.883771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:18.883771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.883771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:18.883771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:18.883771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:18.883771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.884017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:18.884017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:18.884017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:18.884017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:18.884017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:18.884017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:18.884017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:18.884017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:18.884017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.884017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:18.884017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:18.884017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.884017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:18.884251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.884251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:18.885537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:18.886262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:18.886755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:18.886755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:18.886755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:21.159024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:21.159851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:21.159851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:21.160966 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:21.196926 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:21.197202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:21.197992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:21.197992 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:21.197992 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:21.197992 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:21.197992 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:21.197992 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:21.197992 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:21.197992 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:21.198958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:21.198958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:21.198958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:21.198958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:21.198958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:21.198958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:21.198958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:21.198958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:21.198958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:21.198958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:21.198958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:21.198958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:21.199564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:21.199564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:21.199564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:21.199564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:21.199564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:21.199564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:21.199564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:21.199564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:21.199564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:21.199564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:21.199564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:21.199564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:21.199564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:21.200121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:21.200822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:21.202656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:21.202656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:21.202656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:21.202656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:21.202656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:21.202656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:23.853193 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:23.853857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:23.854029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:23.855226 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:23.855581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:23.855817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.856707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:23.856707 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.856707 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:23.856707 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:23.856707 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.856707 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:23.856707 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.856707 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:23.858228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.858228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:23.858228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:23.858228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:23.858228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:23.858228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:23.858228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:23.858228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.858228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:23.858228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:23.858228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:23.858228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.858875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:23.858875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:23.858875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:23.858875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:23.858875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:23.858875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:23.858875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:23.858875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:23.858875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.858875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:23.858875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:23.858875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.858875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:23.859558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.859558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:23.859892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:23.860542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.861197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:23.861197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.861197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:23.863125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:23.863415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:23.863527 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:23.863645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:55:23.864139 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:23.864403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.864985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:23.864985 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.864985 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:23.864985 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:23.864985 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.864985 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:23.864985 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.864985 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:23.865733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:23.866270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:23.866270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:23.866270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.866270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:23.866270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:23.866270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.866270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:23.866270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.866270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:23.866576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.866576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:23.866576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:23.866576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:23.866576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:23.866576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:23.866576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:23.866576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:55:23.866576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:23.866576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:23.866576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.866576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:23.866963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:23.867401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.867401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:23.867401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.867401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:23.867673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:23.868273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:23.868865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.868865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:23.868865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:23.868865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.868865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:23.868865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.868865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:23.870281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:23.870566 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:23.870677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:23.870798 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:23.871302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:23.871642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.872287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:23.872287 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.872287 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:23.872287 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:23.872287 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.872287 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:23.872287 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.872287 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:23.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:23.873506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:23.873506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:23.873506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.873506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:23.873506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:23.873506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.873506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:23.873506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.873506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:23.873893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.873893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:23.873893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:23.873893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:23.873893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:23.873893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:23.873893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:23.873893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:23.873893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:23.873893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.873893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:23.874269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:23.874658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.874658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:23.874658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.874658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:23.874936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:23.875467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:23.876004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.876004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:23.876004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.876004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:23.880171 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:23.880375 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:23.880483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:23.880648 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:23.880976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:23.881369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:23.881544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.882037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:23.882037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.882037 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:23.882037 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:23.882037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.882037 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:23.882037 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.882037 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:23.882849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.882849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:23.882849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:23.882849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:23.882849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:23.882849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:23.882849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:23.882849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.882849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:23.882849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:23.882849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:23.882849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:23.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:23.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:23.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:23.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:23.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:23.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:23.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:23.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:23.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:23.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:23.883637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.883637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:23.883795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:23.884280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:23.884845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:23.884845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:23.884845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:26.159658 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:26.160383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:26.160383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:26.161477 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:26.198695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:26.199065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:26.200092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:26.200092 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:26.200092 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:26.200092 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:26.200092 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:26.200092 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:26.200092 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:26.200092 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:26.201163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:26.201163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:26.201163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:26.201163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:26.201163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:26.201163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:26.201163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:26.201163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:26.201163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:26.201163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:26.201163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:26.201163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:26.201980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:26.201980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:26.201980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:26.201980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:26.201980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:26.201980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:26.201980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:26.201980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:26.201980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:26.201980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:26.201980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:26.201980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:26.201980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:26.202649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:26.203650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:26.204737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:26.204737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:26.204737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:26.204737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:26.204737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:26.204737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:28.853772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:28.854409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:28.854409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:28.855189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:28.855557 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:28.855829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.856601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:28.856601 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.856601 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:28.856601 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:28.856601 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.856601 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:28.856601 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.856601 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:28.857464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.857464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:28.857464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:28.857464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:28.857464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:28.857464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:28.857464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:28.857464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.857464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:28.857464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:28.857464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:28.857464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.857850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:28.857850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:28.857850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:28.857850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:28.857850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:28.857850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:28.857850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:28.857850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:28.857850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.857850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:28.857850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:28.857850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.857850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:28.858270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.858270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:28.858560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:28.859385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.860570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:28.860570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.860570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:28.863475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:28.863678 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:28.863800 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:28.863925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:55:28.864403 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:28.864730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.865373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:28.865373 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.865373 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:28.865373 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:28.865373 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.865373 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:28.865373 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.865373 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:28.867062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:28.868022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:28.868774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:28.868774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:28.868774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:55:28.868774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:28.868774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:28.868774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.868774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:28.868774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:28.868774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:28.868774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.868774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:28.868774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:28.868774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:28.869407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:28.869407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:28.869407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:28.869407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:28.869407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:28.869407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.869407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:28.869407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:28.869407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.869407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:28.869407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.869407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:28.869989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:28.870728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:28.871194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.871194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:28.871194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:28.871194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.871194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:28.871194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.871194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:28.871194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:28.871194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:28.871194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:28.871535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:28.871657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:28.872075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.872907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:28.872907 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.872907 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:28.872907 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:28.872907 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.872907 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:28.872907 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.872907 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:28.873783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:28.874169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:28.874169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:28.874169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.874169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:28.874169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:28.874169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.874169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:28.874169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.874169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:28.874834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.874834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:28.874834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:28.874834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:28.874834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:28.874834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:28.874834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:28.874834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:28.874834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:28.874834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.874834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:28.875190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:28.875633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.875633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:28.875633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.875633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:28.876191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:28.876795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:28.877448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.877448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:28.877448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.877448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:28.880447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:28.880651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:28.880778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:28.880955 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:28.881423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:28.881762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:28.881974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.882652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:28.882652 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.882652 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:28.882652 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:28.882652 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.882652 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:28.882652 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.882652 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:28.883620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.883620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:28.883620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:28.883620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:28.883620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:28.883620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:28.883620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:28.883620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.883620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:28.883620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:28.883620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:28.883620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.883980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:28.883980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:28.883980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:28.883980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:28.883980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:28.883980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:28.883980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:28.883980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:28.883980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.883980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:28.883980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:28.883980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.883980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:28.884353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.884353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:28.884862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:28.885313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:28.885812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:28.885812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:28.885812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:31.160305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:31.161069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:31.161069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:31.162025 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:31.198005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:31.198305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:31.199043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:31.199043 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:31.199043 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:31.199043 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:31.199043 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:31.199043 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:31.199043 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:31.199043 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:31.200192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:31.200192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:31.200192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:31.200192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:31.200192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:31.200192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:31.200192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:31.200192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:31.200192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:31.200192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:31.200192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:31.200192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:31.200823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:31.200823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:31.200823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:31.200823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:31.200823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:31.200823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:31.200823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:31.200823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:31.200823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:31.200823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:31.200823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:31.200823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:31.200823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:31.201459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:31.202240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:31.203072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:31.203072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:31.203072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:31.203072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:31.203072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:31.203072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:33.854504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:33.855490 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:33.855490 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:33.856808 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:33.857238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:33.857506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.858387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:33.858387 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.858387 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:33.858387 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:33.858387 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.858387 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:33.858387 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.858387 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:33.859508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.859508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:33.859508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:33.859508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:33.859508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:33.859508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:33.859508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:33.859508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.859508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:33.859508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:33.859508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:33.859508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:33.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:33.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:33.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:33.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:33.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:33.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:33.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:33.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:33.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:33.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.860084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:33.860529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:33.861029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.862298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:33.862298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:33.862298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.862298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:33.862298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.862298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:33.863792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:33.863993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:33.864097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:33.864225 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:55:33.864768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:33.865092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.865857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:33.865857 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.865857 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:33.865857 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:33.865857 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.865857 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:33.865857 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.865857 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:33.866694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:33.867820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:33.868523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:33.868523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:33.868523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:55:33.868523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:33.868523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:33.868523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.868523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:33.868523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:33.868523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:33.868523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.868523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:33.868523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:33.868523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:33.869100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:33.869100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:33.869100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:33.869100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:33.869100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:33.869100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.869100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:33.869100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:33.869100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.869100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:33.869100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.869100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:33.869642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:33.870390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:33.871239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.871239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:33.871239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:33.871239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.871239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:33.871239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.871239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:33.871541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:33.871862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:33.872005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:33.872174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:33.872653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:33.873064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.873730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:33.873730 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.873730 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:33.873730 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:33.873730 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.873730 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:33.873730 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.873730 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:33.874602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:33.875076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:33.875076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:33.875076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.875076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:33.875076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:33.875076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.875076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:33.875076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.875076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:33.875779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.875779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:33.875779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:33.875779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:33.875779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:33.875779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:33.875779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:33.875779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:33.875779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:33.875779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.875779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:33.876108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:33.876512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.876512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:33.876512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.876512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:33.877137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:33.877642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:33.878136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.878136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:33.878136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.878136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:33.880876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:33.881200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:33.881362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:33.881561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:33.882073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:33.882529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:33.882787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.883905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:33.883905 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.883905 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:33.883905 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:33.883905 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.883905 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:33.883905 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.883905 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:33.885859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.885859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:33.885859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:33.885859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:33.885859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:33.885859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:33.885859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:33.885859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.885859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:33.885859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:33.885859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:33.885859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.886238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:33.886238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:33.886238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:33.886238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:33.886238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:33.886238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:33.886238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:33.886238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:33.886238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.886238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:33.886238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:33.886238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.886238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:33.886578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.886578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:33.887662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:33.888369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:33.889019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:33.889019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:33.889019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:36.160761 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:36.161575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:36.161575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:36.161999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:36.198351 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:36.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:36.199618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:36.199618 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:36.199618 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:36.199618 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:36.199618 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:36.199618 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:36.199618 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:36.199618 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:36.200365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:36.200365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:36.200365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:36.200365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:36.200365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:36.200365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:36.200365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:36.200365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:36.200365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:36.200365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:36.200365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:36.200365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:36.200743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:36.200743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:36.200743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:36.200743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:36.200743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:36.200743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:36.200743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:36.200743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:36.200743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:36.200743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:36.200743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:36.200743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:36.200743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:36.201085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:36.201678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:36.202254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:36.202254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:36.202254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:36.202254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:36.202254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:36.202254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:38.855456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:38.856337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:38.856337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:38.856626 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:38.856955 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:38.857187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.857952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:38.857952 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.857952 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:38.857952 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:38.857952 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.857952 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:38.857952 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.857952 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:38.859432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.859432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:38.859432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:38.859432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:38.859432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:38.859432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:38.859432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:38.859432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.859432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:38.859432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:38.859432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:38.859432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.860093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:38.860093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:38.860093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:38.860093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:38.860093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:38.860093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:38.860093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:38.860093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:38.860093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.860093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:38.860093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:38.860093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.860093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:38.860742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:38.861516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.862772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:38.862772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:38.862772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.862772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:38.862772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.862772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:38.864184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:38.864500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:38.864612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:38.864730 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:55:38.865051 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:38.865311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.865823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:38.865823 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.865823 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:38.865823 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:38.865823 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.865823 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:38.865823 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.865823 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:38.866623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:38.867152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:38.867152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:38.867152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.867152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:38.867152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:38.867152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.867152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:38.867152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.867152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:38.867944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.867944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:38.867944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:38.867944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:38.867944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:38.867944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:38.867944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:38.867944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:55:38.867944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:38.867944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:38.867944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.867944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:38.868520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:38.869305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:38.869741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:38.870380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:38.870380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:38.870380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:38.870380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:38.870380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:38.870380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.870380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:38.870380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:38.870380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.870380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:38.870380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.870380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:38.871969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:38.872267 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:38.872380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:38.872509 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:38.872823 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:38.873045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.873563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:38.873563 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.873563 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:38.873563 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:38.873563 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.873563 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:38.873563 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.873563 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:38.874365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:38.874803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:38.874803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:38.874803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.874803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:38.874803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:38.874803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.874803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:38.874803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.874803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:38.875268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.875268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:38.875268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:38.875268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:38.875268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:38.875268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:38.875268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:38.875268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:38.875268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:38.875268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.875268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:38.875607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:38.875989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.875989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:38.875989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.875989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:38.876640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:38.877136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:38.877697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.877697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:38.877697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.877697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:38.881442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:38.881723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:38.881838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:38.882035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:38.882367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:38.882576 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:38.882712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.883188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:38.883188 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.883188 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:38.883188 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:38.883188 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.883188 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:38.883188 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.883188 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:38.884050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.884050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:38.884050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:38.884050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:38.884050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:38.884050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:38.884050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:38.884050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.884050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:38.884050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:38.884050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:38.884050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.884442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:38.884442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:38.884442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:38.884442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:38.884442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:38.884442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:38.884442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:38.884442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:38.884442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.884442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:38.884442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:38.884442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.884442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:38.884916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.884916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:38.885049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:38.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:38.886085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:38.886085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:38.886085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:41.161406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:41.162287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:41.162287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:41.163361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:41.200242 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:41.200481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:41.200938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:41.200938 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:41.200938 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:41.200938 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:41.200938 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:41.200938 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:41.200938 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:41.200938 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:41.201734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:41.201734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:41.201734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:41.201734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:41.201734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:41.201734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:41.201734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:41.201734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:41.201734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:41.201734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:41.201734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:41.201734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:41.202161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:41.202161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:41.202161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:41.202161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:41.202161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:41.202161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:41.202161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:41.202161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:41.202161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:41.202161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:41.202161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:41.202161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:41.202161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:41.202576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:41.202982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:41.203555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:41.203555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:41.203555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:41.203555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:41.203555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:41.203555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:43.856007 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:43.856898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:43.856898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:43.857975 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:43.858327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:43.858606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.859484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:43.859484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.859484 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:43.859484 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:43.859484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.859484 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:43.859484 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.859484 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:43.860919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.860919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:43.860919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:43.860919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:43.860919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:43.860919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:43.860919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:43.860919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.860919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:43.860919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:43.860919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:43.860919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.861690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:43.861690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:43.861690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:43.861690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:43.861690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:43.861690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:43.861690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:43.861690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:43.861690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.861690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:43.861690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:43.861690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.861690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:43.863103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:43.863970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.865341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:43.865341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:43.865341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.865341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:43.865341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.865341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:43.865341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:43.865341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:43.865341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:43.865341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:55:43.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:43.865913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.866503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:43.866503 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.866503 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:43.866503 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:43.866503 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.866503 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:43.866503 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.866503 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:43.867105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:43.867600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:43.867600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:43.867600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.867600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:43.867600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:43.867600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.867600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:43.867600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.867600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:43.868055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.868055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:43.868055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:43.868055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:43.868055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:43.868055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:43.868055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:43.868055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:55:43.868055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:43.868055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:43.868055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.868055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:43.868449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:43.868817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.868817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:43.868817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.868817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:43.869029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:43.869460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:43.870067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.870067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:43.870067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:43.870067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.870067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:43.870067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.870067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:43.872453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:43.872771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:43.872879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:43.873036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:43.873357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:43.873574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.874024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:43.874024 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.874024 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:43.874024 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:43.874024 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.874024 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:43.874024 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.874024 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:43.874719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:43.875093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:43.875093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:43.875093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.875093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:43.875093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:43.875093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.875093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:43.875093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.875093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:43.875571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.875571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:43.875571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:43.875571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:43.875571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:43.875571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:43.875571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:43.875571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:43.875571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:43.875571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.875571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:43.875908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:43.876281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.876281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:43.876281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.876281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:43.876622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:43.877085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:43.877552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.877552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:43.877552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.877552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:43.881944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:43.882356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:43.882492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:43.882647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:43.882945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:43.883173 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:43.883344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.883834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:43.883834 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.883834 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:43.883834 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:43.883834 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.883834 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:43.883834 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.883834 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:43.884624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.884624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:43.884624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:43.884624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:43.884624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:43.884624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:43.884624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:43.884624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.884624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:43.884624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:43.884624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:43.884624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.885265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:43.885265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:43.885265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:43.885265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:43.885265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:43.885265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:43.885265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:43.885265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:43.885265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.885265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:43.885265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:43.885265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.885265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:43.885880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:43.886549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:43.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:43.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:43.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:43.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:43.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:43.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:45.772556 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:45.773452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:45.773452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:45.782701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 07 15:55:45.782893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105503) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 07 15:55:45.783058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:55:45.813321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, resource={'id': '8b456c85-5e2d-47f8-9886-ebf65932cf32', 'name': 'private', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4a00c537-ed4f-405c-b241-82e7bc2cf3fa', '9fc89727-343d-403a-acd7-db6b4950e7b1'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:25:31Z', 'updated_at': '2026-05-07T15:25:34Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 42799}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.813778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.814441 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-b0f8152d-626b-440e-9383-919ec905ab38, external_ids_update=None, columns={'tag': 42799}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.814588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.815446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:45.815446 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:45.815446 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:45.815446 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:45.815446 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:45.815446 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:45.815446 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.815446 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:45.816473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.816473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:45.816473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:55:45.816473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:55:45.816473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:55:45.816473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:55:45.816473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:55:45.816473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:45.816473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:45.816473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:45.816473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:45.817061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:45.817681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:45.817681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.817681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:45.817681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:55:45.844060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, resource={'id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'name': 'public', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6aa26064-0a62-428b-a8ed-fef52fc73984', 'adf79775-4dfa-4110-bca0-59be9560299c'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-07T15:25:37Z', 'updated_at': '2026-05-07T15:25:45Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.844363 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.844833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-2b8b3cbc-130c-4894-94e1-ab0729ec7077, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.845136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.845729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:45.845729 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:45.845729 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:45.845729 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:45.845729 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:45.845729 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:45.845729 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.845729 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:45.846926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.846926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:45.846926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:55:45.846926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:55:45.846926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:55:45.846926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:55:45.846926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:55:45.846926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:45.846926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:45.846926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:45.846926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:45.847418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:45.848015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:45.848015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.848015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:45.848015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:55:45.875002 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, resource={'id': '71aea053-d575-4d13-afc5-e6241ad6a15f', 'name': 'admin_net', 'tenant_id': '74d9c701d5cb4de6a15233057b01e413', 'project_id': '74d9c701d5cb4de6a15233057b01e413', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8e8a2cf7-97a7-4d62-ae4a-fa6c2139e811'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:30:51Z', 'updated_at': '2026-05-07T15:30:52Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 51150}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.875354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.875687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d5042c49-4c90-465a-b356-c18cf966a58b, external_ids_update=None, columns={'tag': 51150}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.875802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.876489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:45.876489 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:45.876489 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:45.876489 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:45.876489 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:45.876489 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:45.876489 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.876489 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:45.877655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.877655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:45.877655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:55:45.877655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:55:45.877655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:55:45.877655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:55:45.877655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:55:45.877655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:45.877655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:45.877655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:45.877655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:45.878171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:45.878816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:45.878816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.878816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:45.878816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:55:45.911254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, resource={'id': '4fb571ad-ed2f-4edf-a199-cb52ff142698', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'tenant_id': '91e1e016e0f8434890282eb48ed58760', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2e6720c7-3159-41b9-80a2-fa6cf6a18808'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:31:26Z', 'updated_at': '2026-05-07T15:31:27Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 29129}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.911638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.912176 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13b8a284-1038-44ba-86e4-667f771dfae2, external_ids_update=None, columns={'tag': 29129}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.912389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.913453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:45.913453 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:45.913453 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:45.913453 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:45.913453 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:45.913453 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:45.913453 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.913453 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:45.914717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.914717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:45.914717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:55:45.914717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:55:45.914717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:55:45.914717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:55:45.914717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 15:55:45.914717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:45.914717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:45.914717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:45.914717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:45.915244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:45.916056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:45.916056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.916056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:45.916056 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:55:45.988124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, resource={'id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'enable_snat': True}, 'gw_port_id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-jseci', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'ip_address': '172.24.5.213', 'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984'}, {'ip_address': '2001:db8::15a', 'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 3, 'tenant_id': '91e1e016e0f8434890282eb48ed58760'}, resource_type=routers, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.988752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.989201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.989595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:45.989997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.991105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:45.991105 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:45.991105 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:45.991105 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:45.991105 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:45.991105 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:45.991105 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.991105 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:45.991634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Unable to update router 0938bfb8-1052-49e2-bd69-e5fda45e8a2f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.993150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.993150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:45.993150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:55:45.993150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:55:45.993150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:55:45.993150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:55:45.993150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 07 15:55:45.993150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 07 15:55:45.993150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 07 15:55:45.993150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 07 15:55:45.993150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:45.993495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:45.993872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:45.993872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:45.993872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:45.993872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:45.993872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:45.993872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:45.993872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:45.993872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:45.993872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:45.993872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 15:55:46.118368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, resource={'id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'name': '', 'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'project_id': '', 'mac_address': 'fa:16:3e:f6:db:65', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000172501', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:46.118603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '6aa26064-0a62-428b-a8ed-fef52fc73984 adf79775-4dfa-4110-bca0-59be9560299c', 'neutron:network_name': 'neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.213/24', '2001:db8::15a/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:46.118893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=b30ffafe-8df3-4064-bcbf-e3d1daad619c, lrouter_port=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:46.119111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:46.119822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:46.119822 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:46.119822 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:46.119822 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:46.119822 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:46.119822 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:46.119822 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:46.119822 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:46.121954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:46.121954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:46.121954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 15:55:46.121954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 15:55:46.121954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 15:55:46.121954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 15:55:46.121954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 07 15:55:46.121954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:46.121954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:46.121954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:46.121954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:46.122630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:46.123390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:46.123390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:46.123390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:46.123390 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_for_inconsistencies finished in 0.349 seconds
May 07 15:55:46.162171 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:46.162876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:46.163044 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:46.164323 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:46.203641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:46.203781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:46.204416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:46.204416 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:46.204416 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:46.204416 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:46.204416 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:46.204416 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:46.204416 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:46.204416 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:46.205430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:46.205430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:46.205430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:46.205430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:46.205430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:46.205430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:46.205430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:46.205430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:46.205430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:46.205430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:46.205430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:46.205430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:46.206070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:46.206070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:46.206070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:46.206070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:46.206070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:46.206070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:46.206070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:46.206070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:46.206070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:46.206070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:46.206070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:46.206070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:46.206070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:46.206630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:46.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:46.208123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:46.208123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:46.208123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:46.208123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:46.208123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:46.208123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:48.856735 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:48.857574 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:48.857574 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:48.859361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:48.859361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:48.859361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.860097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:48.860097 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.860097 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:48.860097 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:48.860097 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.860097 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:48.860097 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.860097 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:48.861635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.861635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:48.861635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:48.861635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:48.861635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:48.861635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:48.861635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:48.861635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.861635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:48.861635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:48.861635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:48.861635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.862301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:48.862301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:48.862301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:48.862301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:48.862301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:48.862301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:48.862301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:48.862301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:48.862301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.862301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:48.862301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:48.862301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.862301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:48.862858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.862858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:48.863245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:48.863988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.864827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:48.864827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.864827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:48.865314 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:48.865566 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:48.865711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:48.865838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:55:48.866291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:48.866584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.867340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:48.867340 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.867340 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:48.867340 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:48.867340 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.867340 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:48.867340 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.867340 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:48.868081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:48.868776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:48.868776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:48.868776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.868776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:48.868776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:48.868776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.868776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:48.868776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.868776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:48.869070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.869070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:48.869070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:48.869070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:48.869070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:48.869070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:48.869070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:48.869070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:55:48.869070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:48.869070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:48.869070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.869070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:48.869454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:48.869839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.869839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:48.869839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.869839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:48.869960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:48.870442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:48.870944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.870944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:48.870944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:48.870944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.870944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:48.870944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.870944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:48.872924 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:48.873219 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:48.873340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:48.873469 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:48.873960 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:48.874379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.875074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:48.875074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.875074 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:48.875074 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:48.875074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.875074 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:48.875074 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.875074 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:48.875851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:48.876350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:48.876350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:48.876350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.876350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:48.876350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:48.876350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.876350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:48.876350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.876350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:48.876759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.876759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:48.876759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:48.876759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:48.876759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:48.876759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:48.876759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:48.876759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:48.876759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:48.876759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.876759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:48.877122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:48.877524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.877524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:48.877524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.877524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:48.877852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:48.878374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:48.878905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.878905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:48.878905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.878905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:48.882471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:48.882749 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:48.882859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:48.883020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:48.883485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:48.883847 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:48.884129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.884837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:48.884837 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.884837 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:48.884837 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:48.884837 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.884837 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:48.884837 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.884837 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:48.885705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.885705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:48.885705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:48.885705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:48.885705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:48.885705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:48.885705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:48.885705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.885705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:48.885705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:48.885705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:48.885705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:48.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:48.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:48.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:48.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:48.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:48.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:48.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:48.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:48.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:48.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:48.886514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.886514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:48.886707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:48.887170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:48.887724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:48.887724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:48.887724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:51.163029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:51.163830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:51.163830 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:51.165001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:51.198394 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:51.199376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:51.200047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:51.200047 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:51.200047 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:51.200047 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:51.200047 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:51.200047 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:51.200047 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:51.200047 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:51.201277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:51.201277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:51.201277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:51.201277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:51.201277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:51.201277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:51.201277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:51.201277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:51.201277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:51.201277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:51.201277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:51.201277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:51.201559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:51.201559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:51.201559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:51.201559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:51.201559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:51.201559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:51.201559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:51.201559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:51.201559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:51.201559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:51.201559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:51.201559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:51.201559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:51.201792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:51.201792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:51.202671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:51.203018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:51.203394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:51.203394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:51.203394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:53.857305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:53.858280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:53.858280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:53.859469 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:53.859943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:53.860300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.861085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:53.861085 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.861085 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:53.861085 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:53.861085 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.861085 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:53.861085 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.861085 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:53.862102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.862102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:53.862102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:53.862102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:53.862102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:53.862102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:53.862102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:53.862102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.862102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:53.862102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:53.862102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:53.862102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.862823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:53.862823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:53.862823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:53.862823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:53.862823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:53.862823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:53.862823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:53.862823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:53.862823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.862823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:53.862823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:53.862823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.862823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:53.863526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:53.864222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.864774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:53.864774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:53.864774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.864774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:53.864774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.864774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:53.865597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:53.865890 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:53.866003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:53.866121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:55:53.866638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:53.867313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.867855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:53.867855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.867855 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:53.867855 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:53.867855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.867855 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:53.867855 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.867855 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:53.868584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:53.869047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:53.869047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:53.869047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.869047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:53.869047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:53.869047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.869047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:53.869047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.869047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:53.869367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.869367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:53.869367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:53.869367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:53.869367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:53.869367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:53.869367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:53.869367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:55:53.869367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:53.869367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:53.869367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.869367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:53.869874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:53.870332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.870332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:53.870332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.870332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:53.870621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:53.871144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:53.871741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.871741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:53.871741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:53.871741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.871741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:53.871741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.871741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:53.873491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:53.873726 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:53.873824 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:53.873957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:53.874353 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:53.874612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.875145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:53.875145 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.875145 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:53.875145 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:53.875145 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.875145 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:53.875145 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.875145 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:53.875957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:53.876942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:53.877455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:53.877455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:53.877455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:53.877455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:53.877455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.877455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:53.877455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:53.877455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:53.877455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.877455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:53.877455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:53.877455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:53.877455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:53.877930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:53.877930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:53.877930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:53.877930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:53.877930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.877930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:53.877930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:53.877930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.877930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:53.877930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.877930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:53.877930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:53.878354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.878916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:53.882797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:53.883023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:53.883145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:53.883383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:53.883705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:53.883945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:53.884105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.884660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:53.884660 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.884660 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:53.884660 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:53.884660 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.884660 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:53.884660 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.884660 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:53.885469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.885469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:53.885469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:53.885469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:53.885469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:53.885469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:53.885469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:53.885469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.885469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:53.885469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:53.885469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:53.885469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.885822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:53.885822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:53.885822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:53.885822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:53.885822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:53.885822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:53.885822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:53.885822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:53.885822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.885822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:53.885822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:53.885822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.885822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:53.886137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.886137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:53.886429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:53.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:53.887383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:53.887383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:53.887383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:56.163689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:56.164530 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:56.164530 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:56.165777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:56.195868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:56.196311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:56.197167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:56.197167 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:56.197167 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:56.197167 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:56.197167 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:56.197167 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:56.197167 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:56.197167 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:56.197946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:56.197946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:56.197946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:56.197946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:56.197946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:56.197946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:56.197946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:56.197946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:56.197946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:56.197946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:56.197946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:56.197946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:56.198569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:56.198569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:56.198569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:56.198569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:56.198569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:56.198569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:56.198569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:56.198569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:56.198569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:56.198569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:56.198569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:56.198569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:56.198569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:56.199160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:56.199847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:56.200762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:56.200762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:56.200762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:56.200762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:56.200762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:56.200762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:58.858057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:58.858900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:58.858999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:58.861336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:58.861336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:58.861336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.861645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:58.861645 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.861645 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:58.861645 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:58.861645 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.861645 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:58.861645 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.861645 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:58.863393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.863393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:58.863393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:58.863393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:58.863393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:58.863393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:58.863393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:58.863393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.863393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:58.863393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:58.863393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:58.863393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.863735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:58.863735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:58.863735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:58.863735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:58.863735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:58.863735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:58.863735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:58.863735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:58.863735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.863735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:58.863735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:58.863735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.863735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:58.864120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.864120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:58.865371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:58.865806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.866350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:58.866350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.866350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:58.866350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:58.866350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:58.866518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:58.866602 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:55:58.866976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:58.867303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.867954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:58.867954 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.867954 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:58.867954 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:58.867954 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.867954 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:58.867954 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.867954 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:58.868902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:58.869381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:58.869381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:58.869381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.869381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:58.869381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:58.869381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.869381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:58.869381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.869381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:58.869777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.869777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:58.869777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:58.869777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:58.869777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:58.869777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:58.869777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:58.869777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:55:58.869777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:58.869777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:58.869777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.869777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:58.870122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:58.870593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.870593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:58.870593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.870593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:58.870847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:58.871293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:58.871799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.871799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:58.871799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:58.871799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.871799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:58.871799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.871799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:58.873777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:58.874108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:58.874328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:58.874464 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:58.875281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:58.875281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.875700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:58.875700 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.875700 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:58.875700 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:58.875700 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.875700 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:58.875700 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.875700 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:55:58.876720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:55:58.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:58.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:55:58.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:55:58.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:55:58.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:55:58.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:55:58.878009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.878009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:58.878009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:58.878009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:58.878009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:58.878009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:58.878009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:58.878009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:58.878009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:58.878009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.878009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:58.878561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:58.879150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.879150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:58.879150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.879150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:58.879514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:58.880034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:58.880570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.880570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:58.880570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.880570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:55:58.883070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:55:58.883389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:55:58.883546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:55:58.883754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:55:58.884220 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:58.884606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:55:58.884822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:55:58.885538 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.885538 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:55:58.885538 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:55:58.885538 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.885538 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:55:58.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:55:58.886537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.886537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:55:58.886537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:58.886537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:55:58.886537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:55:58.886537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:58.886537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:58.886537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.886537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:58.886537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:58.886537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:55:58.886537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.887054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:55:58.887054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:58.887054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:55:58.887054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:58.887054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:55:58.887054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:55:58.887054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:58.887054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:55:58.887054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.887054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:55:58.887054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:55:58.887054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.887054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:55:58.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:55:58.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:55:58.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:55:58.888909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:55:58.888909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:55:58.888909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:01.164191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:01.164859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:01.164991 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:01.166325 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:01.200027 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:01.200497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:01.201512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:01.201512 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:01.201512 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:01.201512 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:01.201512 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:01.201512 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:01.201512 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:01.201512 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:01.203034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:01.203034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:01.203034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:01.203034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:01.203034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:01.203034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:01.203034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:01.203034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:01.203034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:01.203034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:01.203034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:01.203034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:01.203441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:01.203441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:01.203441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:01.203441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:01.203441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:01.203441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:01.203441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:01.203441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:01.203441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:01.203441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:01.203441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:01.203441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:01.203441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:01.203889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:01.204315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:01.204818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:01.204818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:01.204818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:01.204818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:01.204818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:01.204818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:03.859092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:03.859814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:03.859814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:03.860781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:03.861170 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:03.861538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.862460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:03.862460 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.862460 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:03.862460 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:03.862460 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.862460 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:03.862460 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.862460 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:03.863629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.863629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:03.863629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:03.863629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:03.863629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:03.863629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:03.863629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:03.863629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.863629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:03.863629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:03.863629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:03.863629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.864178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:03.864178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:03.864178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:03.864178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:03.864178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:03.864178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:03.864178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:03.864178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:03.864178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.864178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:03.864178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:03.864178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.864178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:03.864775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:03.865438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.866300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:03.866300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:03.866300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.866300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:03.866300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.866300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:03.866552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:03.866696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:03.866806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:03.866932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:56:03.867772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:03.868667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.869024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:03.869024 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.869024 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:03.869024 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:03.869024 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.869024 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:03.869024 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.869024 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:03.869891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:03.870356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:03.870356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:03.870356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.870356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:03.870356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:03.870356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.870356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:03.870356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.870356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:03.870785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.870785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:03.870785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:03.870785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:03.870785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:03.870785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:03.870785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:03.870785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:56:03.870785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:03.870785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:03.870785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.870785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:03.871127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:03.871556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.871556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:03.871556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.871556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:03.871850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:03.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:03.872806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.872806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:03.872806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:03.872806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.872806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:03.872806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.872806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:03.874293 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:03.874569 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:03.874682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:03.874817 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:03.875321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:03.875872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:03.876754 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.876754 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:03.876754 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:03.876754 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.876754 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:03.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:03.877566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:03.877987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:03.877987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:03.877987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.877987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:03.877987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:03.877987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.877987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:03.877987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.877987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:03.878555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.878555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:03.878555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:03.878555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:03.878555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:03.878555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:03.878555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:03.878555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:03.878555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:03.878555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.878555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:03.878878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:03.879284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.879284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:03.879284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.879284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:03.879683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:03.880142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.880674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:03.883624 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:03.883911 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:03.884026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:03.884249 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:03.884832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:03.885120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:03.885318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.885961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:03.885961 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.885961 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:03.885961 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:03.885961 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.885961 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:03.885961 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.885961 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:03.886931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.886931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:03.886931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:03.886931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:03.886931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:03.886931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:03.886931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:03.886931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.886931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:03.886931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:03.886931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:03.886931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.887292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:03.887292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:03.887292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:03.887292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:03.887292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:03.887292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:03.887292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:03.887292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:03.887292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.887292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:03.887292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:03.887292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.887292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:03.887674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.887674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:03.887955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:03.888423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:03.888958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:03.888958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:03.888958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:06.165087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:06.165879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:06.165879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:06.166993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:06.200752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:06.201175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:06.202399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:06.202399 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:06.202399 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:06.202399 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:06.202399 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:06.202399 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:06.202399 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:06.202399 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:06.203196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:06.203196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:06.203196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:06.203196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:06.203196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:06.203196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:06.203196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:06.203196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:06.203196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:06.203196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:06.203196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:06.203196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:06.203732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:06.203732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:06.203732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:06.203732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:06.203732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:06.203732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:06.203732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:06.203732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:06.203732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:06.203732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:06.203732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:06.203732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:06.203732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:06.204346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:06.205048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:06.205829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:06.205829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:06.205829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:06.205829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:06.205829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:06.205829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:08.859745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:08.860592 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:08.860592 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:08.861775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:08.862162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:08.862442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.863474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:08.863474 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.863474 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:08.863474 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:08.863474 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.863474 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:08.863474 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.863474 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:08.865540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.865540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:08.865540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:08.865540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:08.865540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:08.865540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:08.865540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:08.865540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.865540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:08.865540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:08.865540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:08.865540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.866001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:08.866001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:08.866001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:08.866001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:08.866001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:08.866001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:08.866001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:08.866001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:08.866001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.866001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:08.866001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:08.866001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.866001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:08.866427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.866427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:08.867842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:08.868524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.869166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:08.869166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.869166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:08.869166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:08.869166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:08.869166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:08.869166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:56:08.869431 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:08.869729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.870733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:08.870733 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.870733 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:08.870733 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:08.870733 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.870733 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:08.870733 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.870733 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:08.871759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:08.872206 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:08.872206 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:08.872206 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.872206 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:08.872206 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:08.872206 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.872206 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:08.872206 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.872206 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:08.872661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.872661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:08.872661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:08.872661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:08.872661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:08.872661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:08.872661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:08.872661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:56:08.872661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:08.872661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:08.872661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.872661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:08.873018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:08.873390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.873390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:08.873390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.873390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:08.873720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:08.874139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:08.874639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.874639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:08.874639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:08.874639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.874639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:08.874639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.874639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:08.874824 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:08.874975 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:08.875088 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:08.875214 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:08.875818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:08.876152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.876769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:08.876769 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.876769 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:08.876769 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:08.876769 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.876769 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:08.876769 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.876769 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:08.877517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:08.877915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:08.877915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:08.877915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.877915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:08.877915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:08.877915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.877915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:08.877915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.877915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:08.878431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.878431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:08.878431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:08.878431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:08.878431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:08.878431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:08.878431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:08.878431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:08.878431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:08.878431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.878431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:08.878785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:08.879170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.879170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:08.879170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.879170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:08.879770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:08.880295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:08.880802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.880802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:08.880802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.880802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:08.884106 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:08.884391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:08.884503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:08.884657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:08.885398 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:08.885790 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:08.886016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.886796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:08.886796 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.886796 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:08.886796 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:08.886796 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.886796 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:08.886796 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.886796 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:08.887640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.887640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:08.887640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:08.887640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:08.887640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:08.887640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:08.887640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:08.887640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.887640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:08.887640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:08.887640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:08.887640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.887986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:08.887986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:08.887986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:08.887986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:08.887986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:08.887986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:08.887986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:08.887986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:08.887986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.887986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:08.887986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:08.887986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.887986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:08.888359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.888359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:08.888682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:08.889121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:08.889662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:08.889662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:08.889662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:11.165661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:11.166515 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:11.166515 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:11.167052 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:11.172317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=435 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:56:11.203706 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:11.204249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:11.205237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:11.205237 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:11.205237 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:11.205237 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:11.205237 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:11.205237 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:11.205237 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:11.205237 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:11.206780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:11.206780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:11.206780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:11.206780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:11.206780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:11.206780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:11.206780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:11.206780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:11.206780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:11.206780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:11.206780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:11.206780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:11.207513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:11.207513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:11.207513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:11.207513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:11.207513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:11.207513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:11.207513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:11.207513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:11.207513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:11.207513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:11.207513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:11.207513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:11.207513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:11.208179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:11.208778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:11.209311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:11.209311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:11.209311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:11.209311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:11.209311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:11.209311 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:13.860428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:13.860968 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:13.861146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:13.862265 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:13.862610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:13.862844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.863586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:13.863586 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.863586 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:13.863586 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:13.863586 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.863586 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:13.863586 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.863586 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:13.864340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.864340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:13.864340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:13.864340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:13.864340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:13.864340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:13.864340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:13.864340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.864340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:13.864340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:13.864340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:13.864340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:13.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:13.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:13.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:13.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:13.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:13.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:13.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:13.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:13.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:13.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.864702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:13.865028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.865028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:13.865187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:13.866430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.867126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:13.867126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.867126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:13.868593 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:13.868865 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:13.868980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:13.869103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:56:13.869463 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:13.869667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:13.870158 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.870158 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:13.870158 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:13.870158 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.870158 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:13.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:13.870948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:13.871350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:13.871350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:13.871350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.871350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:13.871350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:13.871350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.871350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:13.871350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.871350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:13.871801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.871801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:13.871801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:13.871801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:13.871801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:13.871801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:13.871801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:13.871801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:56:13.871801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:13.871801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:13.871801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.871801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:13.872185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:13.872599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.872599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:13.872599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.872599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:13.872852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:13.873274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:13.873725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.873725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:13.873725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:13.873725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.873725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:13.873725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.873725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:13.875129 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:13.875411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:13.875523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:13.875652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:13.876091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:13.876411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.876973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:13.876973 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.876973 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:13.876973 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:13.876973 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.876973 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:13.876973 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.876973 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:13.877790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:13.878289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:13.878289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:13.878289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.878289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:13.878289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:13.878289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.878289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:13.878289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.878289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:13.878693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.878693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:13.878693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:13.878693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:13.878693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:13.878693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:13.878693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:13.878693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:13.878693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:13.878693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.878693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:13.879011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:13.879399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.879399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:13.879399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.879399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:13.879780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:13.880266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:13.880769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.880769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:13.880769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.880769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:13.884553 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:13.884825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:13.884938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:13.885097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:13.885816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:13.886141 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:13.886387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.887083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:13.887083 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.887083 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:13.887083 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:13.887083 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.887083 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:13.887083 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.887083 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:13.887938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.887938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:13.887938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:13.887938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:13.887938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:13.887938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:13.887938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:13.887938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.887938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:13.887938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:13.887938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:13.887938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.888333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:13.888333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:13.888333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:13.888333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:13.888333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:13.888333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:13.888333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:13.888333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:13.888333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.888333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:13.888333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:13.888333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.888333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:13.888689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.888689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:13.888931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:13.889385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:13.889907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:13.889907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:13.889907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:16.166632 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:16.167575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:16.167575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:16.168692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:16.203121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:16.203426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:16.204174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:16.204174 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:16.204174 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:16.204174 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:16.204174 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:16.204174 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:16.204174 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:16.204174 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:16.205261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:16.205261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:16.205261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:16.205261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:16.205261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:16.205261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:16.205261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:16.205261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:16.205261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:16.205261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:16.205261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:16.205261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:16.205592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:16.205592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:16.205592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:16.205592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:16.205592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:16.205592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:16.205592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:16.205592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:16.205592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:16.205592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:16.205592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:16.205592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:16.205592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:16.205919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:16.205919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:16.206356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:16.206778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:16.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:16.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:16.207277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:18.860954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:18.861760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:18.861760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:18.862600 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:18.862922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:18.863154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.864073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:18.864073 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.864073 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:18.864073 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:18.864073 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.864073 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:18.864073 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.864073 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:18.865553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.865553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:18.865553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:18.865553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:18.865553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:18.865553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:18.865553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:18.865553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.865553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:18.865553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:18.865553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:18.865553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.866531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:18.866531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:18.866531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:18.866531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:18.866531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:18.866531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:18.866531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:18.866531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:18.866531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.866531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:18.866531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:18.866531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.866531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:18.867460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:18.868628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.871299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:18.871299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:18.871299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.871299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:18.871299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.871299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:18.871299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:18.871299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:18.871299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:18.871299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:56:18.872122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:18.872122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.872122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:18.872122 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.872122 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:18.872122 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:18.872122 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.872122 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:18.872122 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.872122 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:18.872761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:18.873461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:18.873461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:18.873461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.873461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:18.873461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:18.873461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.873461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:18.873461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.873461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:18.873863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.873863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:18.873863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:18.873863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:18.873863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:18.873863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:18.873863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:18.873863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:56:18.873863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:18.873863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:18.873863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.873863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:18.874397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:18.874911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:18.875510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:18.876302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:18.876888 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:18.876888 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:18.876888 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:18.876888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.877088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:18.877088 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.877088 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:18.877088 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:18.877088 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.877088 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:18.877088 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.877088 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:18.877835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:18.878496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:18.878496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:18.878496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.878496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:18.878496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:18.878496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.878496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:18.878496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.878496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:18.878839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.878839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:18.878839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:18.878839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:18.878839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:18.878839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:18.878839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:18.878839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:18.878839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:18.878839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.878839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:18.879335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:18.879854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:18.880500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:18.881070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:18.881070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:18.881070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:18.881070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:18.881070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.881070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:18.881070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:18.881070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.881070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:18.881070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.881070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:18.884941 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:18.885180 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:18.885309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:18.885486 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:18.885896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:18.886288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:18.886510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.887157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:18.887157 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.887157 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:18.887157 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:18.887157 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.887157 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:18.887157 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.887157 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:18.888073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.888073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:18.888073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:18.888073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:18.888073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:18.888073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:18.888073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:18.888073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.888073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:18.888073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:18.888073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:18.888073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.888534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:18.888534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:18.888534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:18.888534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:18.888534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:18.888534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:18.888534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:18.888534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:18.888534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.888534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:18.888534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:18.888534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.888534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:18.888906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.888906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:18.889094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:18.889750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:18.890509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:18.890509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:18.890509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:21.167136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:21.167951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:21.167951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:21.168926 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:21.195638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:21.195998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:21.196829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:21.196829 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:21.196829 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:21.196829 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:21.196829 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:21.196829 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:21.196829 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:21.196829 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:21.197607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:21.197607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:21.197607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:21.197607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:21.197607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:21.197607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:21.197607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:21.197607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:21.197607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:21.197607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:21.197607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:21.197607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:21.198225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:21.198225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:21.198225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:21.198225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:21.198225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:21.198225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:21.198225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:21.198225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:21.198225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:21.198225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:21.198225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:21.198225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:21.198225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:21.198813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:21.199518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:21.200366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:21.200366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:21.200366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:21.200366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:21.200366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:21.200366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:23.861622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:23.862232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:23.862232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:23.863181 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:23.863524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:23.863757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.864551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:23.864551 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.864551 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:23.864551 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:23.864551 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.864551 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:23.864551 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.864551 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:23.865625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.865625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:23.865625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:23.865625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:23.865625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:23.865625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:23.865625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:23.865625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.865625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:23.865625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:23.865625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:23.865625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.866026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:23.866026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:23.866026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:23.866026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:23.866026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:23.866026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:23.866026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:23.866026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:23.866026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.866026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:23.866026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:23.866026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.866026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:23.866666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.866666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:23.866931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:23.867579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.868349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:23.868349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.868349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:23.869445 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:23.869737 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:23.869866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:23.869995 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:56:23.870450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:23.870722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.871397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:23.871397 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.871397 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:23.871397 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:23.871397 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.871397 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:23.871397 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.871397 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:23.872168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:23.872778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:23.872778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:23.872778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.872778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:23.872778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:23.872778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.872778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:23.872778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.872778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:23.873138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.873138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:23.873138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:23.873138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:23.873138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:23.873138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:23.873138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:23.873138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:56:23.873138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:23.873138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:23.873138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.873138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:23.873544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:23.874032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.874032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:23.874032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.874032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:23.874238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:23.874877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:23.875609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.875609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:23.875609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:23.875609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.875609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:23.875609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.875609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:23.876049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:23.876367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:23.876492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:23.876625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:23.877051 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:23.877460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.878100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:23.878100 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.878100 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:23.878100 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:23.878100 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.878100 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:23.878100 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.878100 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:23.878852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:23.879484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:23.879484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:23.879484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.879484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:23.879484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:23.879484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.879484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:23.879484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.879484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:23.879879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.879879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:23.879879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:23.879879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:23.879879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:23.879879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:23.879879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:23.879879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:23.879879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:23.879879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.879879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:23.880357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:23.880946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:23.881586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:23.882386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:23.882386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:23.882386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:23.882386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:23.882386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.882386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:23.882386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:23.882386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.882386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:23.882386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.882386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:23.885444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:23.885694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:23.885811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:23.885983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:23.886424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:23.886744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:23.886950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.887619 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:23.887619 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.887619 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:23.887619 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:23.887619 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.887619 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:23.887619 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.887619 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:23.888513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.888513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:23.888513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:23.888513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:23.888513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:23.888513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:23.888513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:23.888513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.888513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:23.888513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:23.888513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:23.888513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.888992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:23.888992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:23.888992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:23.888992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:23.888992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:23.888992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:23.888992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:23.888992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:23.888992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.888992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:23.888992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:23.888992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.888992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:23.889620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:23.890272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:23.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:23.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:23.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:23.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:23.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:23.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:26.167774 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:26.168273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:26.168402 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:26.169232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:26.197767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:26.198167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:26.199159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:26.199159 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:26.199159 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:26.199159 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:26.199159 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:26.199159 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:26.199159 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:26.199159 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:26.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:26.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:26.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:26.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:26.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:26.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:26.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:26.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:26.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:26.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:26.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:26.199891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:26.200198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:26.200198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:26.200198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:26.200198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:26.200198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:26.200198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:26.200198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:26.200198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:26.200198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:26.200198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:26.200198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:26.200198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:26.200198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:26.200474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:26.200474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:26.200644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:26.200987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:26.201387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:26.201387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:26.201387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:28.862349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:28.863065 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:28.863065 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:28.863949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:28.864288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:28.864555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.865316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:28.865316 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.865316 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:28.865316 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:28.865316 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.865316 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:28.865316 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.865316 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:28.866376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.866376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:28.866376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:28.866376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:28.866376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:28.866376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:28.866376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:28.866376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.866376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:28.866376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:28.866376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:28.866376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.866937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:28.866937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:28.866937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:28.866937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:28.866937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:28.866937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:28.866937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:28.866937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:28.866937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.866937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:28.866937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:28.866937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.866937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:28.867510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:28.868137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.868953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:28.868953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:28.868953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.868953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:28.868953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.868953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:28.870048 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:28.870372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:28.870489 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:28.870620 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:56:28.870978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:28.871271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.871747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:28.871747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.871747 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:28.871747 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:28.871747 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.871747 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:28.871747 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.871747 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:28.872562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:28.873233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:28.873233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:28.873233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.873233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:28.873233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:28.873233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.873233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:28.873233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.873233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:28.873696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.873696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:28.873696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:28.873696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:28.873696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:28.873696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:28.873696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:28.873696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:56:28.873696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:28.873696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:28.873696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.873696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:28.874173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:28.874590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:28.875021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:28.875550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:28.875550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:28.875550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:28.875550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:28.875550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:28.875550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.875550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:28.875550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:28.875550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.875550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:28.875550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.875550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:28.876452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:28.876764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:28.876905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:28.876995 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:28.877354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:28.877603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.878195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:28.878195 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.878195 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:28.878195 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:28.878195 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.878195 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:28.878195 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.878195 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:28.878952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:28.879393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:28.879393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:28.879393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.879393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:28.879393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:28.879393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.879393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:28.879393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.879393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:28.880105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.880105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:28.880105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:28.880105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:28.880105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:28.880105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:28.880105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:28.880105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:28.880105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:28.880105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.880105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:28.880492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:28.880882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.880882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:28.880882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.880882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:28.881222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:28.881763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:28.882324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.882324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:28.882324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.882324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:28.885853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:28.886245 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:28.886354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:28.886529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:28.886850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:28.887094 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:28.887468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.887949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:28.887949 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.887949 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:28.887949 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:28.887949 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.887949 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:28.887949 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.887949 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:28.888823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.888823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:28.888823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:28.888823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:28.888823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:28.888823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:28.888823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:28.888823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.888823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:28.888823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:28.888823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:28.888823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.889172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:28.889172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:28.889172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:28.889172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:28.889172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:28.889172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:28.889172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:28.889172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:28.889172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.889172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:28.889172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:28.889172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.889172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:28.889592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.889592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:28.889856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:28.890322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:28.890862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:28.890862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:28.890862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:31.168488 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:31.169496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:31.169496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:31.170434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:31.208477 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:31.208851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:31.209763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:31.209763 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:31.209763 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:31.209763 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:31.209763 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:31.209763 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:31.209763 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:31.209763 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:31.210787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:31.210787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:31.210787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:31.210787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:31.210787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:31.210787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:31.210787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:31.210787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:31.210787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:31.210787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:31.210787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:31.210787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:31.211587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:31.211587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:31.211587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:31.211587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:31.211587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:31.211587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:31.211587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:31.211587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:31.211587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:31.211587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:31.211587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:31.211587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:31.211587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:31.212344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:31.213283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:31.214196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:31.214196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:31.214196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:31.214196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:31.214196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:31.214196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:33.862920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:33.863450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:33.863501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:33.864476 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:33.864799 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:33.865068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.865936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:33.865936 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.865936 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:33.865936 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:33.865936 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.865936 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:33.865936 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.865936 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:33.866951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.866951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:33.866951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:33.866951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:33.866951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:33.866951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:33.866951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:33.866951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.866951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:33.866951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:33.866951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:33.866951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.867552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:33.867552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:33.867552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:33.867552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:33.867552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:33.867552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:33.867552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:33.867552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:33.867552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.867552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:33.867552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:33.867552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.867552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:33.868107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:33.868765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.869652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:33.869652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:33.869652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.869652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:33.869652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.869652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:33.870623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:33.870953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:33.871076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:33.871236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:56:33.871676 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:33.871955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.874199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:33.874199 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.874199 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:33.874199 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:33.874199 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.874199 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:33.874199 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.874199 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:33.875162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:33.875787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:33.875787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:33.875787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.875787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:33.875787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:33.875787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.875787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:33.875787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.875787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:33.876189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.876189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:33.876189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:33.876189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:33.876189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:33.876189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:33.876189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:33.876189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:56:33.876189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:33.876189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:33.876189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.876189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:33.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:33.877304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.877304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:33.877304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.877304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:33.877457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:33.878003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:33.878546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.878546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:33.878546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:33.878546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.878546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:33.878546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.878546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:33.878546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:33.878546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:33.878546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:33.878859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:33.878859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:33.879080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.879792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:33.879792 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.879792 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:33.879792 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:33.879792 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.879792 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:33.879792 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.879792 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:33.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:33.881153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:33.881153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:33.881153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.881153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:33.881153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:33.881153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.881153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:33.881153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.881153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:33.881635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.881635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:33.881635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:33.881635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:33.881635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:33.881635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:33.881635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:33.881635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:33.881635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:33.881635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.881635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:33.881971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:33.882402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.882402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:33.882402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.882402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:33.882721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:33.883272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:33.883809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.883809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:33.883809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.883809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:33.886356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:33.886586 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:33.886670 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:33.886787 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:33.887163 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:33.887521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:33.887754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.888371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:33.888371 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.888371 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:33.888371 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:33.888371 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.888371 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:33.888371 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.888371 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:33.888949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.888949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:33.888949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:33.888949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:33.888949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:33.888949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:33.888949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:33.888949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.888949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:33.888949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:33.888949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:33.888949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.889792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:33.889792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:33.889792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:33.889792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:33.889792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:33.889792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:33.889792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:33.889792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:33.889792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.889792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:33.889792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:33.889792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.889792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:33.890123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:33.890535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:33.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:33.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:33.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:33.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:33.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:33.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:36.169093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:36.170065 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:36.170293 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:36.171521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:36.205978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:36.206189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:36.206788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:36.206788 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:36.206788 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:36.206788 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:36.206788 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:36.206788 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:36.206788 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:36.206788 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:36.207730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:36.207730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:36.207730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:36.207730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:36.207730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:36.207730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:36.207730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:36.207730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:36.207730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:36.207730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:36.207730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:36.207730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:36.208335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:36.208335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:36.208335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:36.208335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:36.208335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:36.208335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:36.208335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:36.208335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:36.208335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:36.208335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:36.208335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:36.208335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:36.208335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:36.208922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:36.209584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:36.210405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:36.210405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:36.210405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:36.210405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:36.210405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:36.210405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:38.863647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:38.864405 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:38.864405 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:38.865480 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:38.865842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:38.866088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.866884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:38.866884 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.866884 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:38.866884 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:38.866884 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.866884 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:38.866884 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.866884 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:38.867930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.867930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:38.867930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:38.867930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:38.867930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:38.867930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:38.867930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:38.867930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.867930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:38.867930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:38.867930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:38.867930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.868525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:38.868525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:38.868525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:38.868525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:38.868525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:38.868525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:38.868525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:38.868525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:38.868525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.868525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:38.868525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:38.868525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.868525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:38.869057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:38.869685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.870463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:38.870463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:38.870463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.870463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:38.870463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.870463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:38.871227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:38.871589 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:38.871721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:38.872555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:56:38.872555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:38.872555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.873068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:38.873068 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.873068 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:38.873068 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:38.873068 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.873068 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:38.873068 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.873068 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:38.873817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:38.874251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:38.874251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:38.874251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.874251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:38.874251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:38.874251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.874251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:38.874251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.874251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:38.876532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.876532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:38.876532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:38.876532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:38.876532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:38.876532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:38.876532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:38.876532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:56:38.876532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:38.876532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:38.876532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.876532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:38.876921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:38.877323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:38.877716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:38.878259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:38.878259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:38.878259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:38.878259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:38.878259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:38.878259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.878259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:38.878259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:38.878259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.878259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:38.878259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.878259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:38.878259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:38.878635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:38.878635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:38.878635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:38.878969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:38.879200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.879755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:38.879755 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.879755 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:38.879755 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:38.879755 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.879755 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:38.879755 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.879755 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:38.880540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:38.880828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:38.880828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:38.880828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.880828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:38.880828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:38.880828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.880828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:38.880828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.880828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:38.881399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.881399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:38.881399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:38.881399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:38.881399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:38.881399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:38.881399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:38.881399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:38.881399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:38.881399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.881399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:38.881631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:38.881901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.881901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:38.881901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.881901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:38.882443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:38.882766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:38.883091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.883091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:38.883091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.883091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:38.886692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:38.886938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:38.887049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:38.887188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:38.887522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:38.887747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:38.887892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.888457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:38.888457 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.888457 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:38.888457 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:38.888457 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.888457 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:38.888457 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.888457 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:38.889280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.889280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:38.889280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:38.889280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:38.889280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:38.889280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:38.889280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:38.889280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.889280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:38.889280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:38.889280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:38.889280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.889832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:38.889832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:38.889832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:38.889832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:38.889832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:38.889832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:38.889832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:38.889832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:38.889832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.889832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:38.889832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:38.889832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.889832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:38.890427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:38.891054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:38.891891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:38.891891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:38.891891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:38.891891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:38.891891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:38.891891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:41.170095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:41.173370 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:41.173370 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:41.173370 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:41.211478 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:41.211833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:41.212472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:41.212472 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:41.212472 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:41.212472 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:41.212472 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:41.212472 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:41.212472 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:41.212472 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:41.213224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:41.213224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:41.213224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:41.213224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:41.213224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:41.213224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:41.213224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:41.213224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:41.213224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:41.213224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:41.213224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:41.213224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:41.213663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:41.213663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:41.213663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:41.213663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:41.213663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:41.213663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:41.213663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:41.213663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:41.213663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:41.213663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:41.213663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:41.213663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:41.213663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:41.214125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:41.214691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:41.215414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:41.215414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:41.215414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:41.215414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:41.215414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:41.215414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:43.864360 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:43.865160 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:43.865160 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:43.866261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:43.866595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:43.867658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.868688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:43.868688 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.868688 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:43.868688 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:43.868688 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.868688 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:43.868688 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.868688 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:43.870392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.870392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:43.870392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:43.870392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:43.870392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:43.870392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:43.870392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:43.870392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.870392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:43.870392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:43.870392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:43.870392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.870960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:43.870960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:43.870960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:43.870960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:43.870960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:43.870960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:43.870960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:43.870960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:43.870960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.870960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:43.870960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:43.870960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.870960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:43.871704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.871704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:43.872060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:43.872980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.873811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:43.873811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.873811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:43.873811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:43.873811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:43.873811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:43.873811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:56:43.873811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:43.874304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.875469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:43.875469 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.875469 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:43.875469 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:43.875469 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.875469 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:43.875469 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.875469 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:43.875969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:43.876889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:43.876889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:43.876889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.876889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:43.876889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:43.876889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.876889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:43.876889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.876889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:43.877796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.877796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:43.877796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:43.877796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:43.877796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:43.877796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:43.877796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:43.877796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:56:43.877796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:43.877796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:43.877796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.877796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:43.878457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:43.878834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.878834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:43.878834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.878834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:43.880167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:43.880660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:43.881309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.881309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:43.881309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:43.881309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.881309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:43.881309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.881309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:43.881309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:43.881309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:43.881309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:43.881764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:43.881764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:43.881764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.882185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:43.882185 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.882185 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:43.882185 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:43.882185 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.882185 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:43.882185 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.882185 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:43.883031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:43.883589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:43.883589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:43.883589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.883589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:43.883589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:43.883589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.883589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:43.883589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.883589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:43.884898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.884898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:43.884898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:43.884898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:43.884898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:43.884898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:43.884898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:43.884898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:43.884898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:43.884898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.884898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:43.885461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:43.885844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:43.886435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:43.886974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:43.886974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:43.886974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:43.886974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:43.886974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.886974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:43.886974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:43.886974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.886974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:43.886974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.886974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:43.887659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:43.887659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:43.887659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:43.887868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:43.888178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:43.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:43.888587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.889132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:43.889132 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.889132 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:43.889132 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:43.889132 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.889132 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:43.889132 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.889132 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:43.890162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.890162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:43.890162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:43.890162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:43.890162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:43.890162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:43.890162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:43.890162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.890162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:43.890162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:43.890162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:43.890162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.890896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:43.890896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:43.890896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:43.890896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:43.890896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:43.890896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:43.890896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:43.890896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:43.890896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.890896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:43.890896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:43.890896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.890896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:43.891568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.891568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:43.891677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:43.892239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:43.892904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:43.892904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:43.892904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:46.171721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:46.172677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:46.172677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:46.173984 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:46.209062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:46.209519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:46.210479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:46.210479 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:46.210479 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:46.210479 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:46.210479 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:46.210479 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:46.210479 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:46.210479 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:46.211410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:46.211410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:46.211410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:46.211410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:46.211410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:46.211410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:46.211410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:46.211410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:46.211410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:46.211410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:46.211410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:46.211410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:46.211777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:46.211777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:46.211777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:46.211777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:46.211777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:46.211777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:46.211777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:46.211777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:46.211777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:46.211777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:46.211777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:46.211777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:46.211777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:46.212133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:46.212133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:46.212457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:46.212796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:46.213140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:46.213140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:46.213140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:48.865075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:48.865891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:48.865891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:48.866929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:48.867351 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:48.867708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.868753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:48.868753 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.868753 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:48.868753 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:48.868753 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.868753 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:48.868753 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.868753 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:48.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:48.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:48.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:48.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:48.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:48.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:48.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:48.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:48.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:48.870158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.870513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:48.870513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:48.870513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:48.870513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:48.870513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:48.870513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:48.870513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:48.870513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:48.870513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.870513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:48.870513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:48.870513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.870513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:48.870823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.870823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:48.871987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:48.872536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.872966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:48.872966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.872966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:48.872966 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:48.873072 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:48.873425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:48.873504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:56:48.873847 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:48.874037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.874655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:48.874655 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.874655 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:48.874655 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:48.874655 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.874655 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:48.874655 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.874655 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:48.875888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:48.876767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:48.876767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:48.876767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.876767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:48.876767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:48.876767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.876767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:48.876767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.876767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:48.877333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.877333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:48.877333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:48.877333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:48.877333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:48.877333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:48.877333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:48.877333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:56:48.877333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:48.877333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:48.877333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.877333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:48.877614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:48.877931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.877931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:48.877931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.877931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:48.878910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:48.879319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:48.879724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.879724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:48.879724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:48.879724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.879724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:48.879724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.879724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:48.880702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:48.880957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:48.881062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:48.881191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:48.881481 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:48.881693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.882114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:48.882114 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.882114 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:48.882114 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:48.882114 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.882114 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:48.882114 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.882114 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:48.882873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:48.883245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:48.883245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:48.883245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.883245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:48.883245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:48.883245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.883245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:48.883245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.883245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:48.883716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.883716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:48.883716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:48.883716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:48.883716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:48.883716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:48.883716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:48.883716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:48.883716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:48.883716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.883716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:48.884343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.884343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:48.884343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.884343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:48.884751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:48.885159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:48.885586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.885586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:48.885586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.885586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:48.887498 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:48.887779 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:48.887882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:48.888036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:48.888294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:48.888498 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:48.888625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.889024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:48.889024 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.889024 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:48.889024 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:48.889024 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.889024 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:48.889024 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.889024 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:48.889973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.889973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:48.889973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:48.889973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:48.889973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:48.889973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:48.889973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:48.889973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.889973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:48.889973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:48.889973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:48.889973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.890314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:48.890314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:48.890314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:48.890314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:48.890314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:48.890314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:48.890314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:48.890314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:48.890314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.890314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:48.890314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:48.890314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.890314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:48.890620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.890620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:48.890987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:48.891404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:48.892310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:48.892310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:48.892310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:51.172237 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:51.173032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:51.173032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:51.174306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:51.209407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:51.209407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:51.211307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:51.212162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:51.212907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:51.213657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:51.214758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:51.214758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:51.214758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:51.214758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:51.214758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:51.214758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:51.214758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:51.214758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:51.214758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:53.865730 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:53.866655 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:53.866655 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:53.867669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:53.868050 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:53.868569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.869237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:53.869237 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.869237 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:53.869237 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:53.869237 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.869237 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:53.869237 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.869237 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:53.870797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.870797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:53.870797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:53.870797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:53.870797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:53.870797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:53.870797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:53.870797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.870797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:53.870797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:53.870797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:53.870797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.871498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:53.871498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:53.871498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:53.871498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:53.871498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:53.871498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:53.871498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:53.871498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:53.871498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.871498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:53.871498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:53.871498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.871498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:53.872786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:53.873762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.874570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:53.874570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:53.874570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.874570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:53.874570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.874570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:53.874570 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:53.874570 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:53.874570 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:53.874570 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:56:53.875094 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:53.875094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.875094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:53.875094 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.875094 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:53.875094 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:53.875094 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.875094 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:53.875094 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.875094 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:53.877814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:53.878837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:53.879465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:53.879465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:53.879465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:56:53.879465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:53.879465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:53.879465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.879465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:53.879465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:53.879465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:53.879465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.879465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:53.879465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:53.879465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:53.880546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:53.880546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:53.880546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:53.880546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:53.880546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:53.880546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.880546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:53.880546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:53.880546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.880546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:53.880546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.880546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:53.880841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:53.881128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:53.881501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.881501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:53.881501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:53.881501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.881501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:53.881501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.881501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:53.881501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:53.881501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:53.881501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:53.881720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:53.882059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:53.882439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.883129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:53.883129 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.883129 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:53.883129 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:53.883129 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.883129 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:53.883129 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.883129 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:53.883846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:53.884120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:53.884120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:53.884120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.884120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:53.884120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:53.884120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.884120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:53.884120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.884120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:53.884682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.884682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:53.884682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:53.884682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:53.884682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:53.884682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:53.884682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:53.884682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:53.884682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:53.884682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.884682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:53.884928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:53.885190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.885190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:53.885190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.885190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:53.885747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:53.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:53.886530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.886530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:53.886530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.886530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:53.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:53.888188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:53.888315 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:53.888482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:53.889338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:53.889338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:53.889476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.890119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:53.890119 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.890119 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:53.890119 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:53.890119 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.890119 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:53.890119 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.890119 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:53.890921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.890921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:53.890921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:53.890921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:53.890921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:53.890921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:53.890921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:53.890921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.890921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:53.890921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:53.890921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:53.890921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.891339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:53.891339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:53.891339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:53.891339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:53.891339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:53.891339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:53.891339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:53.891339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:53.891339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.891339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:53.891339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:53.891339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.891339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:53.891717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.891717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:53.891853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:53.892456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:53.893046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:53.893046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:53.893046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:56.172956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:56.173803 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:56.173985 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:56.175160 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:56.209510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:56.210003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:56.210793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:56.210793 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:56.210793 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:56.210793 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:56.210793 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:56.210793 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:56.210793 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:56.210793 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:56.211539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:56.211539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:56.211539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:56.211539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:56.211539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:56.211539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:56.211539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:56.211539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:56.211539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:56.211539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:56.211539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:56.211539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:56.212127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:56.212127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:56.212127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:56.212127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:56.212127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:56.212127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:56.212127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:56.212127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:56.212127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:56.212127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:56.212127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:56.212127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:56.212127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:56.212765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:56.213477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:56.214322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:56.214322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:56.214322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:56.214322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:56.214322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:56.214322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:58.866414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:58.867172 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:58.867301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:58.868309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:58.868542 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:58.868700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.869498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:58.869498 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.869498 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:58.869498 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:58.869498 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.869498 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:58.869498 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.869498 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:58.870930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.870930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:58.870930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:58.870930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:58.870930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:58.870930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:58.870930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:58.870930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.870930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:58.870930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:58.870930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:58.870930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.871826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:58.871826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:58.871826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:58.871826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:58.871826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:58.871826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:58.871826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:58.871826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:58.871826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.871826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:58.871826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:58.871826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.871826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:58.872682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:58.873756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.875282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:58.875282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:58.875282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.875282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:58.875282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.875282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:58.875282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:58.875282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:58.875282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:58.875282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:56:58.875977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:58.875977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.875977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:58.875977 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.875977 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:58.875977 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:58.875977 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.875977 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:58.875977 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.875977 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:58.876399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:58.876937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:58.876937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:58.876937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.876937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:58.876937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:58.876937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.876937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:58.876937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.876937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:58.877284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.877284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:58.877284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:58.877284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:58.877284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:58.877284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:58.877284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:58.877284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:56:58.877284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:58.877284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:58.877284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.877284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:58.877727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:58.878232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:58.878742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:58.879401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:58.879401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:58.879401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:58.879401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:58.879401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:58.879401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.879401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:58.879401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:58.879401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.879401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:58.879401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.879401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:58.881329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:58.881581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:58.881692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:58.881857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:58.882219 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:58.882481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.882952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:58.882952 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.882952 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:58.882952 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:58.882952 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.882952 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:58.882952 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.882952 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:56:58.883708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:56:58.884255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:58.884255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:56:58.884255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.884255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:56:58.884255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:56:58.884255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.884255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:56:58.884255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.884255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:56:58.884628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.884628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:58.884628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:58.884628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:58.884628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:58.884628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:58.884628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:58.884628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:58.884628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:58.884628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.884628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:58.885057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:58.885545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.885545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:58.885545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.885545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:58.885745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:58.886395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:58.887053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.887053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:58.887053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.887053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:56:58.888414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:56:58.888698 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:56:58.888799 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:56:58.888965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:56:58.889557 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:58.889884 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:56:58.890096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.890751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:56:58.890751 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.890751 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:56:58.890751 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:56:58.890751 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.890751 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:56:58.890751 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.890751 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:56:58.891603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.891603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:56:58.891603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:58.891603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:56:58.891603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:56:58.891603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:58.891603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:58.891603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.891603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:58.891603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:58.891603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:56:58.891603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.892059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:56:58.892059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:58.892059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:56:58.892059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:58.892059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:56:58.892059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:56:58.892059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:58.892059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:56:58.892059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.892059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:56:58.892059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:56:58.892059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.892059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:56:58.892576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.892576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:56:58.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:56:58.893399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:56:58.894064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:56:58.894064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:56:58.894064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:01.174075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:01.174733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:01.174872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:01.176263 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:01.212479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:01.212652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:01.213186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:01.213186 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:01.213186 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:01.213186 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:01.213186 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:01.213186 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:01.213186 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:01.213186 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:01.213820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:01.213820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:01.213820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:01.213820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:01.213820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:01.213820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:01.213820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:01.213820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:01.213820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:01.213820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:01.213820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:01.213820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:01.214488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:01.214488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:01.214488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:01.214488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:01.214488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:01.214488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:01.214488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:01.214488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:01.214488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:01.214488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:01.214488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:01.214488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:01.214488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:01.215139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:01.215950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:01.216808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:01.216808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:01.216808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:01.216808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:01.216808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:01.216808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:03.867149 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:03.867980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:03.867980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:03.868999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:03.869373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:03.869620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.870400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:03.870400 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.870400 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:03.870400 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:03.870400 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.870400 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:03.870400 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.870400 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:03.871507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.871507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:03.871507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:03.871507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:03.871507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:03.871507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:03.871507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:03.871507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.871507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:03.871507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:03.871507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:03.871507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.872053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:03.872053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:03.872053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:03.872053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:03.872053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:03.872053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:03.872053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:03.872053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:03.872053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.872053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:03.872053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:03.872053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.872053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:03.872606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:03.873250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.874071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:03.874071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:03.874071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.874071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:03.874071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.874071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:03.874071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:03.874744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:03.874744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:03.874744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:57:03.875121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:03.875343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.875900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:03.875900 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.875900 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:03.875900 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:03.875900 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.875900 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:03.875900 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.875900 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:03.876686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:03.877410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:03.877410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:03.877410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.877410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:03.877410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:03.877410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.877410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:03.877410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.877410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:03.877821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.877821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:03.877821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:03.877821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:03.877821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:03.877821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:03.877821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:03.877821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:57:03.877821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:03.877821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:03.877821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.877821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:03.878409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:03.878965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:03.879741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:03.880450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:03.880450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:03.880450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:03.880450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:03.880450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:03.880450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.880450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:03.880450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:03.880450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.880450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:03.880450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.880450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:03.881683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:03.881960 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:03.882072 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:03.882201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:03.882726 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:03.883191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.884021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:03.884021 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.884021 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:03.884021 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:03.884021 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.884021 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:03.884021 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.884021 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:03.884781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:03.885250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:03.885250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:03.885250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.885250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:03.885250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:03.885250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.885250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:03.885250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.885250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:03.885686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.885686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:03.885686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:03.885686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:03.885686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:03.885686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:03.885686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:03.885686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:03.885686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:03.885686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.885686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:03.886055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:03.886492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.886492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:03.886492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.886492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:03.887106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:03.887652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:03.888178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.888178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:03.888178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.888178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:03.888814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:03.889054 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:03.889168 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:03.889339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:03.889783 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:03.890163 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:03.890340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.890820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:03.890820 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.890820 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:03.890820 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:03.890820 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.890820 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:03.890820 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.890820 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:03.891690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.891690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:03.891690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:03.891690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:03.891690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:03.891690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:03.891690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:03.891690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.891690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:03.891690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:03.891690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:03.891690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.892042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:03.892042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:03.892042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:03.892042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:03.892042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:03.892042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:03.892042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:03.892042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:03.892042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.892042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:03.892042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:03.892042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.892042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:03.892423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.892423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:03.892704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:03.893200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:03.894061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:03.894061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:03.894061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:06.174811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:06.175653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:06.175653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:06.176969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:06.207936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:06.208965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:06.208965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:06.208965 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:06.208965 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:06.208965 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:06.208965 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:06.208965 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:06.208965 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:06.208965 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:06.209630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:06.209630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:06.209630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:06.209630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:06.209630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:06.209630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:06.209630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:06.209630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:06.209630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:06.209630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:06.209630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:06.209630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:06.210239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:06.210239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:06.210239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:06.210239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:06.210239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:06.210239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:06.210239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:06.210239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:06.210239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:06.210239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:06.210239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:06.210239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:06.210239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:06.210815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:06.211549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:06.212383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:06.212383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:06.212383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:06.212383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:06.212383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:06.212383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:08.867812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:08.868793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:08.868984 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:08.869888 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:08.870249 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:08.870500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.871340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:08.871340 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.871340 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:08.871340 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:08.871340 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.871340 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:08.871340 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.871340 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:08.872770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.872770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:08.872770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:08.872770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:08.872770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:08.872770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:08.872770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:08.872770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.872770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:08.872770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:08.872770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:08.872770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.873432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:08.873432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:08.873432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:08.873432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:08.873432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:08.873432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:08.873432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:08.873432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:08.873432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.873432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:08.873432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:08.873432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.873432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:08.873999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.873999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:08.874435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:08.875317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.876144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:08.876144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.876144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:08.876144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:08.876144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:08.876144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:08.876144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:57:08.876144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:08.876468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.876823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:08.876823 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.876823 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:08.876823 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:08.876823 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.876823 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:08.876823 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.876823 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:08.880700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:08.881121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:08.881121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:08.881121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.881121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:08.881121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:08.881121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.881121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:08.881121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.881121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:08.881404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.881404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:08.881404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:08.881404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:08.881404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:08.881404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:08.881404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:08.881404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:57:08.881404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:08.881404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:08.881404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.881404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:08.881909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:08.882317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.882317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:08.882317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.882317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:08.882505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:08.883249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:08.883701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.883701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:08.883701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:08.883701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.883701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:08.883701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.883701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:08.883701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:08.883701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:08.883701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:08.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:08.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:08.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:08.884000 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.884000 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:08.884000 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:08.884000 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.884000 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:08.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:08.884325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:08.884725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:08.884725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:08.884725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.884725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:08.884725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:08.884725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.884725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:08.884725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.884725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:08.884990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.884990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:08.884990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:08.884990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:08.884990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:08.884990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:08.884990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:08.884990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:08.884990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:08.884990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.884990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:08.885334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:08.885708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:08.886080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:08.886664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:08.886664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:08.886664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:08.886664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:08.886664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.886664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:08.886664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:08.886664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.886664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:08.886664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.886664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:08.889071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:08.889244 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:08.889331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:08.889443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:08.889606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:08.889759 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:08.889858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.890171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:08.890171 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.890171 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:08.890171 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:08.890171 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.890171 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:08.890171 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.890171 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:08.890984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.890984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:08.890984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:08.890984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:08.890984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:08.890984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:08.890984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:08.890984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.890984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:08.890984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:08.890984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:08.890984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.891367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:08.891367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:08.891367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:08.891367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:08.891367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:08.891367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:08.891367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:08.891367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:08.891367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.891367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:08.891367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:08.891367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.891367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:08.891829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.891829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:08.892012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:08.892596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:08.893288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:08.893288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:08.893288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:11.175428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:11.176462 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:11.176462 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:11.177475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:11.185970 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:57:11.209219 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:11.209553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:11.210228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:11.210228 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:11.210228 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:11.210228 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:11.210228 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:11.210228 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:11.210228 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:11.210228 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:11.211132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:11.211132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:11.211132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:11.211132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:11.211132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:11.211132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:11.211132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:11.211132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:11.211132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:11.211132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:11.211132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:11.211132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:11.211989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:11.211989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:11.211989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:11.211989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:11.211989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:11.211989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:11.211989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:11.211989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:11.211989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:11.211989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:11.211989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:11.211989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:11.211989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:11.212617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:11.213370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:11.214186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:11.214186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:11.214186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:11.214186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:11.214186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:11.214186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:13.868956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:13.870069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:13.870069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:13.870821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:13.871199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:13.871466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.872402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:13.872402 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.872402 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:13.872402 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:13.872402 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.872402 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:13.872402 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.872402 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:13.873943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.873943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:13.873943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:13.873943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:13.873943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:13.873943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:13.873943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:13.873943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.873943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:13.873943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:13.873943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:13.873943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.874338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:13.874338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:13.874338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:13.874338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:13.874338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:13.874338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:13.874338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:13.874338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:13.874338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.874338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:13.874338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:13.874338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.874338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:13.874691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.874691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:13.875615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:13.876067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.876599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:13.876599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.876599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:13.876599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:13.876599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:13.876599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:13.876599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:57:13.876873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:13.877343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.877861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:13.877861 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.877861 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:13.877861 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:13.877861 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.877861 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:13.877861 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.877861 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:13.878639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:13.879867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:13.880877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:13.880877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:13.880877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:57:13.880877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:13.880877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:13.880877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.880877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:13.880877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:13.880877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:13.880877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.880877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:13.880877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:13.880877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:13.881337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:13.881337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:13.881337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:13.881337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:13.881337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:13.881337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.881337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:13.881337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:13.881337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.881337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:13.881337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.881337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:13.881869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:13.882745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:13.883341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.883341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:13.883341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:13.883341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.883341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:13.883341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.883341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:13.883341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:13.883341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:13.883341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:13.883869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:13.883869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:13.884340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.884567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:13.884567 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.884567 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:13.884567 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:13.884567 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.884567 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:13.884567 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.884567 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:13.885337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:13.886385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:13.887001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:13.887001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:13.887001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:13.887001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:13.887001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.887001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:13.887001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:13.887001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:13.887001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.887001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:13.887001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:13.887001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:13.887001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:13.887418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:13.887418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:13.887418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:13.887418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:13.887418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.887418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:13.887418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:13.887418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.887418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:13.887418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.887418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:13.887418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:13.887651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.888009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:13.889390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:13.889630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:13.889731 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:13.889896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:13.890176 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:13.890405 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:13.890541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.891043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:13.891043 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.891043 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:13.891043 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:13.891043 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.891043 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:13.891043 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.891043 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:13.891910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.891910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:13.891910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:13.891910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:13.891910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:13.891910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:13.891910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:13.891910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.891910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:13.891910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:13.891910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:13.891910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.892631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:13.892631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:13.892631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:13.892631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:13.892631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:13.892631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:13.892631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:13.892631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:13.892631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.892631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:13.892631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:13.892631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.892631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:13.892893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.892893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:13.893106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:13.893466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:13.893800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:13.893800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:13.893800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:16.175988 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:16.176839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:16.176839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:16.177853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:16.214714 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:16.215177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:16.215916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:16.215916 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:16.215916 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:16.215916 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:16.215916 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:16.215916 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:16.215916 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:16.215916 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:16.216570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:16.216570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:16.216570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:16.216570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:16.216570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:16.216570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:16.216570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:16.216570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:16.216570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:16.216570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:16.216570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:16.216570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:16.216995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:16.216995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:16.216995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:16.216995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:16.216995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:16.216995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:16.216995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:16.216995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:16.216995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:16.216995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:16.216995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:16.216995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:16.216995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:16.217477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:16.217947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:16.218547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:16.218547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:16.218547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:16.218547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:16.218547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:16.218547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:18.869751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:18.870857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:18.871056 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:18.871984 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:18.872271 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:18.872436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.873357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:18.873357 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.873357 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:18.873357 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:18.873357 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.873357 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:18.873357 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.873357 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:18.874900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.874900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:18.874900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:18.874900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:18.874900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:18.874900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:18.874900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:18.874900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.874900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:18.874900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:18.874900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:18.874900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.875590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:18.875590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:18.875590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:18.875590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:18.875590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:18.875590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:18.875590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:18.875590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:18.875590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.875590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:18.875590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:18.875590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.875590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:18.876099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.876099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:18.876754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:18.877454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.878236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:18.878236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.878236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:18.878236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:18.878236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:18.878236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:18.878236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:57:18.878236 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:18.878685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.879891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:18.879891 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.879891 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:18.879891 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:18.879891 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.879891 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:18.879891 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.879891 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:18.881107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:18.883362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:18.883362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:18.883362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.883362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:18.883362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:18.883362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.883362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:18.883362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.883362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:18.883678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.883678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:18.883678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:18.883678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:18.883678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:18.883678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:18.883678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:18.883678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:57:18.883678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:18.883678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:18.883678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.883678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:18.884000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:18.884516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.884516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:18.884516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.884516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:18.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:18.886672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:18.887334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.887334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:18.887334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:18.887334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.887334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:18.887334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.887334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:18.887334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:18.887334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:18.887334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:18.887613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:18.887613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:18.887889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.888535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:18.888535 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.888535 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:18.888535 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:18.888535 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.888535 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:18.888535 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.888535 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:18.889769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:18.890635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:18.890635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:18.890635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.890635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:18.890635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:18.890635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.890635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:18.890635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.890635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:18.891530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.891530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:18.891530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:18.891530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:18.891530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:18.891530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:18.891530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:18.891530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:18.891530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:18.891530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.891530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:18.891846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:18.892178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.892178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:18.892178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.892178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:18.892178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:18.893994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:18.895094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:18.896058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.896058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:18.896058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.896058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:18.896058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:18.896058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:18.896058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:18.896058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:18.896319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:18.896319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.896319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:18.896319 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.896319 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:18.896319 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:18.896319 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.896319 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:18.896319 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.896319 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:18.896852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.896852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:18.896852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:18.896852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:18.896852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:18.896852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:18.896852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:18.896852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.896852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:18.896852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:18.896852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:18.896852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.897575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:18.897575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:18.897575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:18.897575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:18.897575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:18.897575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:18.897575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:18.897575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:18.897575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.897575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:18.897575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:18.897575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.897575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:18.898118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:18.898604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:18.899128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:18.899128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:18.899128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:18.899128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:18.899128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:18.899128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:21.176770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:21.177598 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:21.177598 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:21.179199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:21.208362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:21.208583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:21.208993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:21.208993 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:21.208993 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:21.208993 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:21.208993 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:21.208993 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:21.208993 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:21.208993 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:21.209737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:21.209737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:21.209737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:21.209737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:21.209737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:21.209737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:21.209737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:21.209737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:21.209737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:21.209737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:21.209737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:21.209737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:21.210650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:21.210650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:21.210650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:21.210650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:21.210650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:21.210650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:21.210650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:21.210650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:21.210650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:21.210650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:21.210650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:21.210650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:21.210650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:21.211241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:21.211828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:21.212611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:21.212611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:21.212611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:21.212611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:21.212611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:21.212611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:23.870864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:23.871495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:23.871698 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:23.873465 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:23.873748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:23.873986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.875028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:23.875028 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.875028 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:23.875028 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:23.875028 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.875028 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:23.875028 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.875028 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:23.875955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.875955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:23.875955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:23.875955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:23.875955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:23.875955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:23.875955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:23.875955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.875955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:23.875955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:23.875955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:23.875955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.876916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:23.876916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:23.876916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:23.876916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:23.876916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:23.876916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:23.876916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:23.876916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:23.876916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.876916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:23.876916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:23.876916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.876916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:23.877670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:23.879006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.880121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:23.880121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:23.880121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.880121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:23.880121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.880121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:23.880121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:23.880121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:23.880121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:23.880121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:23.880814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.881480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:23.882467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.882467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:23.882467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.882467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:23.882467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:23.882467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:23.882467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:23.882467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:23.882467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:23.882467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:57:23.882467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:23.882467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:23.883167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:23.883782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.884467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.885410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:23.886530 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:23.886887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:23.887039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:23.887172 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:23.887479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:23.887720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.888198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:23.888198 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.888198 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:23.888198 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:23.888198 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.888198 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:23.888198 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.888198 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:23.889049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:23.889389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:23.889389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:23.889389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.889389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:23.889389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:23.889389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.889389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:23.889389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.889389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:23.889987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.889987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:23.889987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:23.889987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:23.889987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:23.889987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:23.889987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:23.889987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:23.889987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:23.889987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.889987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:23.890294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:23.890571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.890571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:23.890571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.890571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:23.891149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:23.891542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:23.891917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.891917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:23.891917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.891917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:23.894570 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:23.894942 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:23.895119 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:23.895385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:23.895691 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:23.895910 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:23.896052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:23.898298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:23.898977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:23.899432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:23.899870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:23.900494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:23.900494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:23.900494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:23.900494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:23.900494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:23.900494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:23.900494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:23.900494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:23.900494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:23.900494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:26.177363 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:26.178007 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:26.178007 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:26.179296 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:26.220647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:26.220896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:26.221623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:26.221623 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:26.221623 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:26.221623 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:26.221623 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:26.221623 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:26.221623 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:26.221623 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:26.222718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:26.222718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:26.222718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:26.222718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:26.222718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:26.222718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:26.222718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:26.222718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:26.222718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:26.222718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:26.222718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:26.222718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:26.223125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:26.223125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:26.223125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:26.223125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:26.223125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:26.223125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:26.223125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:26.223125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:26.223125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:26.223125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:26.223125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:26.223125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:26.223125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:26.223530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:26.223530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:26.223799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:26.224264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:26.224758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:26.224758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:26.224758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:28.871591 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:28.872267 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:28.872471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:28.873661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:28.873988 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:28.874290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.875478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:28.875478 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.875478 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:28.875478 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:28.875478 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.875478 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:28.875478 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.875478 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:28.877293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.877293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:28.877293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:28.877293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:28.877293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:28.877293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:28.877293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:28.877293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.877293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:28.877293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:28.877293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:28.877293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.877937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:28.877937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:28.877937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:28.877937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:28.877937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:28.877937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:28.877937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:28.877937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:28.877937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.877937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:28.877937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:28.877937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.877937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:28.878637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.878637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:28.879961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:28.880910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.882191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:28.882191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.882191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:28.882191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:28.882191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:28.882191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:28.882191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:57:28.882191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:28.882510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.882547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:28.882547 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.882547 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:28.882547 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:28.882547 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.882547 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:28.882547 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.882547 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:28.883325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:28.883740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:28.883740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:28.883740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.883740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:28.883740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:28.883740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.883740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:28.883740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.883740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:28.884233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.884233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:28.884233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:28.884233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:28.884233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:28.884233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:28.884233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:28.884233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:57:28.884233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:28.884233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:28.884233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.884233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:28.884923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:28.885788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:28.886200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:28.886713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:28.886713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:28.886713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:28.886713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:28.886713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:28.886713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.886713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:28.886713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:28.886713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.886713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:28.886713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.886713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:28.887128 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:28.887577 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:28.887577 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:28.887577 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:28.888029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:28.888406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.889090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:28.889090 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.889090 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:28.889090 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:28.889090 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.889090 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:28.889090 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.889090 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:28.889875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:28.890728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:28.890728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:28.890728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.890728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:28.890728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:28.890728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.890728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:28.890728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.890728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:28.891227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.891227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:28.891227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:28.891227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:28.891227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:28.891227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:28.891227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:28.891227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:28.891227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:28.891227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.891227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:28.891852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:28.892408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:28.892854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:28.893349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:28.893349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:28.893349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:28.893349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:28.893349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.893349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:28.893349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:28.893349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.893349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:28.893349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.893349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:28.894912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:28.895161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:28.895291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:28.895447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:28.895833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:28.896159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:28.896408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.897081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:28.897081 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.897081 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:28.897081 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:28.897081 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.897081 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:28.897081 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.897081 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:28.898013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.898013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:28.898013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:28.898013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:28.898013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:28.898013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:28.898013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:28.898013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.898013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:28.898013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:28.898013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:28.898013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.898353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:28.898353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:28.898353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:28.898353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:28.898353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:28.898353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:28.898353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:28.898353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:28.898353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.898353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:28.898353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:28.898353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.898353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:28.898641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.898641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:28.899001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:28.899804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:28.900515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:28.900515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:28.900515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:31.178005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:31.178914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:31.179107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:31.180724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:31.235137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:31.235381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:31.236038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:31.236038 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:31.236038 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:31.236038 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:31.236038 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:31.236038 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:31.236038 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:31.236038 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:31.237124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.057 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:31.237124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:31.237124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:31.237124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:31.237124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:31.237124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:31.237124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:31.237124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:31.237124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:31.237124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:31.237124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:31.237124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:31.237647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:31.237647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:31.237647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:31.237647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:31.237647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:31.237647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:31.237647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:31.237647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:31.237647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:31.237647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:31.237647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:31.237647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:31.237647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:31.238301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:31.238301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:31.238394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:31.238824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:31.239370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:31.239370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:31.239370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:33.872317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:33.873352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:33.873352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:33.874439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:33.874693 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:33.874827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.875802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:33.875802 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.875802 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:33.875802 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:33.875802 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.875802 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:33.875802 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.875802 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:33.877321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.877321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:33.877321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:33.877321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:33.877321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:33.877321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:33.877321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:33.877321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.877321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:33.877321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:33.877321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:33.877321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.878040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:33.878040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:33.878040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:33.878040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:33.878040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:33.878040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:33.878040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:33.878040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:33.878040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.878040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:33.878040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:33.878040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.878040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:33.878675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.878675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:33.879051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:33.881519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.883537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:33.883537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.883537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:33.883537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:33.883537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:33.883537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:33.883537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:57:33.883537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:33.883941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.884472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:33.885163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.885163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:33.885163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:33.885163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:33.885163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:33.885163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:33.885163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:33.885163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:57:33.885163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:33.885163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:33.885163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.885163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:33.885728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:33.886436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:33.887563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:33.889091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:33.889536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:33.889536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:33.889536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:33.889536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.889675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:33.889675 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.889675 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:33.889675 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:33.889675 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.889675 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:33.889675 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.889675 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:33.890477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:33.890905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:33.890905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:33.890905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.890905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:33.890905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:33.890905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.890905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:33.890905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.890905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:33.891473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.891473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:33.891473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:33.891473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:33.891473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:33.891473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:33.891473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:33.891473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:33.891473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:33.891473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.891473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:33.892412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:33.893205 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:33.893493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:33.893825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:33.893825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:33.893825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:33.893825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:33.893825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.893825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:33.893825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:33.893825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.893825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:33.893825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.893825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:33.895279 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:33.895578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:33.895692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:33.895856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:33.896261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:33.896588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:33.896823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.897484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:33.897484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.897484 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:33.897484 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:33.897484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.897484 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:33.897484 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.897484 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:33.898352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.898352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:33.898352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:33.898352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:33.898352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:33.898352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:33.898352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:33.898352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.898352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:33.898352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:33.898352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:33.898352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.898602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:33.898602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:33.898602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:33.898602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:33.898602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:33.898602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:33.898602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:33.898602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:33.898602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.898602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:33.898602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:33.898602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.898602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:33.898825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.898825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:33.899367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:33.899678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:33.900003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:33.900003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:33.900003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:36.178682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:36.179423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:36.179555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:36.180867 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:36.221957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:36.222257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:36.222958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:36.222958 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:36.222958 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:36.222958 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:36.222958 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:36.222958 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:36.222958 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:36.222958 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:36.224496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:36.224496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:36.224496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:36.224496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:36.224496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:36.224496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:36.224496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:36.224496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:36.224496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:36.224496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:36.224496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:36.224496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:36.224888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:36.224888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:36.224888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:36.224888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:36.224888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:36.224888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:36.224888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:36.224888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:36.224888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:36.224888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:36.224888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:36.224888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:36.224888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:36.225260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:36.225260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:36.225670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:36.226111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:36.227318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:36.227318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:36.227318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:38.873048 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:38.873776 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:38.873776 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:38.875339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:38.875339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:38.875339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.875631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:38.875631 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.875631 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:38.875631 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:38.875631 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.875631 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:38.875631 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.875631 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:38.876836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.876836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:38.876836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:38.876836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:38.876836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:38.876836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:38.876836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:38.876836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.876836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:38.876836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:38.876836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:38.876836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.877474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:38.877474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:38.877474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:38.877474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:38.877474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:38.877474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:38.877474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:38.877474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:38.877474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.877474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:38.877474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:38.877474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.877474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:38.878051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:38.878790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.879660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:38.879660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:38.879660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.879660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:38.879660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.879660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:38.881463 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:38.882051 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:38.882051 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:38.882249 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:57:38.882715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:38.882996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.883907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:38.883907 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.883907 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:38.883907 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:38.883907 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.883907 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:38.883907 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.883907 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:38.884478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:38.885073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:38.885073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:38.885073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.885073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:38.885073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:38.885073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.885073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:38.885073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.885073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:38.885381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.885381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:38.885381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:38.885381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:38.885381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:38.885381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:38.885381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:38.885381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:57:38.885381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:38.885381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:38.885381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.885381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:38.885734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:38.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:38.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.886098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:38.886740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:38.887869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:38.889422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.889422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:38.889422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:38.889422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.889422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:38.889422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.889422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:38.889422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:38.889422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:38.889422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:38.890388 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:38.890388 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:38.890388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.890388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:38.890388 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.890388 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:38.890388 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:38.890388 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.890388 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:38.890388 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.890388 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:38.890886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:38.891322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:38.891322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:38.891322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.891322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:38.891322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:38.891322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.891322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:38.891322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.891322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:38.891696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.891696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:38.891696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:38.891696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:38.891696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:38.891696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:38.891696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:38.891696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:38.891696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:38.891696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.891696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:38.892257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:38.892712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.892712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:38.892712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.892712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:38.892912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:38.893716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:38.894077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.894077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:38.894077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.894077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:38.895757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:38.896381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:38.896500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:38.896693 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:38.897043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:38.897385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:38.897589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.898180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:38.898180 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.898180 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:38.898180 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:38.898180 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.898180 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:38.898180 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.898180 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:38.898948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.898948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:38.898948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:38.898948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:38.898948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:38.898948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:38.898948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:38.898948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.898948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:38.898948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:38.898948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:38.898948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.899357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:38.899357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:38.899357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:38.899357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:38.899357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:38.899357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:38.899357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:38.899357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:38.899357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.899357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:38.899357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:38.899357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.899357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:38.899699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.899699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:38.899915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:38.900540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:38.901109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:38.901109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:38.901109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:41.179374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:41.180167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:41.180167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:41.181335 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:41.224636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:41.224854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:41.225395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:41.225395 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:41.225395 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:41.225395 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:41.225395 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:41.225395 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:41.225395 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:41.225395 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:41.226446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:41.226446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:41.226446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:41.226446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:41.226446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:41.226446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:41.226446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:41.226446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:41.226446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:41.226446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:41.226446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:41.226446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:41.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:41.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:41.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:41.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:41.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:41.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:41.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:41.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:41.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:41.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:41.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:41.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:41.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:41.227196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:41.227196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:41.227550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:41.228065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:41.228648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:41.228648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:41.228648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:43.874005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:43.874724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:43.874724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:43.875889 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:43.876150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:43.876377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.877073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:43.877073 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.877073 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:43.877073 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:43.877073 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.877073 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:43.877073 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.877073 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:43.878224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.878224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:43.878224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:43.878224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:43.878224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:43.878224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:43.878224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:43.878224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.878224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:43.878224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:43.878224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:43.878224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.878649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:43.878649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:43.878649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:43.878649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:43.878649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:43.878649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:43.878649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:43.878649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:43.878649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.878649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:43.878649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:43.878649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.878649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:43.879079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.879079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:43.879391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:43.880158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.880806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:43.880806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.880806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:43.881984 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:43.882281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:43.882397 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:43.882523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:57:43.883029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:43.883445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.884238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:43.884238 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.884238 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:43.884238 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:43.884238 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.884238 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:43.884238 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.884238 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:43.884951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:43.885593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:43.885593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:43.885593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.885593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:43.885593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:43.885593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.885593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:43.885593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.885593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:43.885864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.885864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:43.885864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:43.885864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:43.885864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:43.885864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:43.885864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:43.885864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:57:43.885864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:43.885864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:43.885864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.885864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:43.886223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:43.886788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:43.887393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:43.888071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:43.888071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:43.888071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:43.888071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:43.888071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:43.888071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.888071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:43.888071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:43.888071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.888071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:43.888071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.888071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:43.888535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:43.888842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:43.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:43.889187 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:43.889787 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:43.890298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.891181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:43.891181 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.891181 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:43.891181 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:43.891181 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.891181 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:43.891181 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.891181 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:43.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:43.892551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:43.892551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:43.892551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.892551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:43.892551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:43.892551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.892551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:43.892551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.892551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:43.892874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.892874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:43.892874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:43.892874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:43.892874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:43.892874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:43.892874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:43.892874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:43.892874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:43.892874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.892874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:43.893339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:43.893841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:43.894273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:43.894898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:43.894898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:43.894898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:43.894898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:43.894898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.894898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:43.894898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:43.894898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.894898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:43.894898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.894898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:43.896705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:43.897239 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:43.897386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:43.897477 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:43.897834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:43.898093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:43.898257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.898817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:43.898817 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.898817 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:43.898817 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:43.898817 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.898817 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:43.898817 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.898817 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:43.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:43.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:43.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:43.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:43.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:43.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:43.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:43.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:43.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:43.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.900360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:43.900360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:43.900360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:43.900360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:43.900360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:43.900360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:43.900360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:43.900360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:43.900360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.900360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:43.900360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:43.900360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.900360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:43.900862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:43.901490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:43.902229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:43.902229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:43.902229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:43.902229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:43.902229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:43.902229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:46.180220 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:46.180832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:46.181025 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:46.182329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:46.216796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:46.217232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:46.218099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:46.218099 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:46.218099 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:46.218099 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:46.218099 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:46.218099 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:46.218099 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:46.218099 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:46.218971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:46.218971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:46.218971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:46.218971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:46.218971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:46.218971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:46.218971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:46.218971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:46.218971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:46.218971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:46.218971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:46.218971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:46.219347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:46.219347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:46.219347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:46.219347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:46.219347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:46.219347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:46.219347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:46.219347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:46.219347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:46.219347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:46.219347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:46.219347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:46.219347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:46.219693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:46.219693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:46.220012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:46.220416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:46.220764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:46.220764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:46.220764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:48.874633 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:48.876131 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:48.876131 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:48.877320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:48.877320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:48.877320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.878152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:48.878152 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.878152 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:48.878152 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:48.878152 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.878152 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:48.878152 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.878152 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:48.879771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.879771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:48.879771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:48.879771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:48.879771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:48.879771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:48.879771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:48.879771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.879771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:48.879771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:48.879771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:48.879771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.880333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:48.880333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:48.880333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:48.880333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:48.880333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:48.880333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:48.880333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:48.880333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:48.880333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.880333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:48.880333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:48.880333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.880333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:48.880757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.880757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:48.881791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:48.882361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.882908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:48.882908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.882908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:48.882908 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:48.883183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:48.883425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:48.883639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:57:48.883983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:48.884169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.884741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:48.884741 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.884741 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:48.884741 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:48.884741 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.884741 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:48.884741 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.884741 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:48.886054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:48.886543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:48.886543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:48.886543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.886543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:48.886543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:48.886543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.886543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:48.886543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.886543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:48.888018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.888018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:48.888018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:48.888018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:48.888018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:48.888018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:48.888018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:48.888018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:57:48.888018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:48.888018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:48.888018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.888018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:48.888420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:48.888751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.888751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:48.888751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.888751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:48.889340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:48.889815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:48.890345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.890345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:48.890345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:48.890345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.890345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:48.890345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.890345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:48.890345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:48.890345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:48.890345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:48.890630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:48.890630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:48.890709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.891235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:48.891235 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.891235 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:48.891235 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:48.891235 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.891235 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:48.891235 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.891235 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:48.892579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:48.892579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:48.892579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.892579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:48.892579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:48.892579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.892579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:48.892579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.892579 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:48.893578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.893578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:48.893578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:48.893578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:48.893578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:48.893578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:48.893578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:48.893578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:48.893578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:48.893578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.893578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:48.893921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:48.894331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.894331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:48.894331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.894331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:48.895444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:48.895950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:48.896491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.896491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:48.896491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.896491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:48.897302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:48.897522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:48.897616 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:48.897732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:48.898014 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:48.898265 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:48.898447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.898876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:48.898876 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.898876 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:48.898876 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:48.898876 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.898876 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:48.898876 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.898876 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:48.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:48.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:48.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:48.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:48.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:48.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:48.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:48.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:48.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:48.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:48.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:48.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:48.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:48.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:48.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:48.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:48.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:48.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:48.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:48.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.899812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:48.900125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:48.900527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:48.901030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:48.901030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:48.901030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:48.901030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:48.901030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:48.901030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:51.181027 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:51.181844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:51.182062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:51.183244 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:51.244011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:51.244417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:51.245227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:51.245227 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:51.245227 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:51.245227 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:51.245227 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:51.245227 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:51.245227 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:51.245227 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:51.246142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:51.246142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:51.246142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:51.246142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:51.246142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:51.246142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:51.246142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:51.246142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:51.246142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:51.246142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:51.246142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:51.246142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:51.246907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:51.246907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:51.246907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:51.246907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:51.246907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:51.246907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:51.246907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:51.246907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:51.246907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:51.246907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:51.246907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:51.246907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:51.246907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:51.247521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:51.248359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:51.249158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:51.249158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:51.249158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:51.249158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:51.249158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:51.249158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:53.875447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:53.876280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:53.876280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:53.877184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:53.877453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:53.877593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.878246 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:53.878246 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.878246 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:53.878246 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:53.878246 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.878246 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:53.878246 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.878246 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:53.879729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.879729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:53.879729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:53.879729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:53.879729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:53.879729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:53.879729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:53.879729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.879729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:53.879729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:53.879729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:53.879729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.880323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:53.880323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:53.880323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:53.880323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:53.880323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:53.880323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:53.880323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:53.880323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:53.880323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.880323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:53.880323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:53.880323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.880323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:53.880886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.880886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:53.881391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:53.882153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.883174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:53.883174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.883174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:53.883174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:53.883483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:53.883574 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:53.883757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:57:53.884060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:53.884234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.884773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:53.884773 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.884773 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:53.884773 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:53.884773 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.884773 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:53.884773 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.884773 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:53.885794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:53.886466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:53.886466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:53.886466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.886466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:53.886466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:53.886466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.886466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:53.886466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.886466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:53.886761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.886761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:53.886761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:53.886761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:53.886761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:53.886761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:53.886761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:53.886761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:57:53.886761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:53.886761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:53.886761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.886761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:53.887111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:53.887488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.887488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:53.887488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.887488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:53.887721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:53.888124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:53.888624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.888624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:53.888624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:53.888624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.888624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:53.888624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.888624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:53.889821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:53.890024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:53.890136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:53.890288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:53.890585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:53.890802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.891250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:53.891250 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.891250 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:53.891250 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:53.891250 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.891250 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:53.891250 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.891250 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:53.891993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:53.892560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:53.892560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:53.892560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.892560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:53.892560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:53.892560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.892560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:53.892560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.892560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:53.892931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.892931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:53.892931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:53.892931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:53.892931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:53.892931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:53.892931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:53.892931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:53.892931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:53.892931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.892931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:53.893267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:53.893695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.893695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:53.893695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.893695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:53.894079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:53.894581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:53.895106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.895106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:53.895106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.895106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:53.897666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:53.897911 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:53.898062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:53.898318 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:53.898585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:53.898832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:53.898889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.899454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:53.899454 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.899454 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:53.899454 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:53.899454 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.899454 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:53.899454 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.899454 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:53.900590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.900590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:53.900590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:53.900590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:53.900590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:53.900590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:53.900590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:53.900590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.900590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:53.900590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:53.900590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:53.900590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.900945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:53.900945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:53.900945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:53.900945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:53.900945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:53.900945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:53.900945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:53.900945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:53.900945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.900945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:53.900945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:53.900945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.900945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:53.901327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.901327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:53.901895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:53.902462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:53.903173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:53.903173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:53.903173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:56.181916 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:56.182685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:56.182685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:56.183643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:56.218930 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:56.219315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:56.222649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:56.223494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:56.224069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:56.224692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:56.225679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:56.225679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:56.225679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:56.225679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:56.225679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:56.225679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:56.225679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:56.225679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:56.225679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:58.876100 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:58.876767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:58.876767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:58.877865 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:58.878227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:58.878493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.879519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:58.879519 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.879519 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:58.879519 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:58.879519 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.879519 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:58.879519 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.879519 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:58.881347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.881347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:58.881347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:58.881347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:58.881347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:58.881347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:58.881347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:58.881347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.881347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:58.881347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:58.881347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:58.881347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.881751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:58.881751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:58.881751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:58.881751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:58.881751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:58.881751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:58.881751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:58.881751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:58.881751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.881751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:58.881751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:58.881751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.881751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:58.882083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.882083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:58.882534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:58.883011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.884226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:58.884226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.884226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:58.884226 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:58.884226 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:58.884226 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:58.884226 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:57:58.884500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:58.884651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.885465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:58.885465 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.885465 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:58.885465 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:58.885465 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.885465 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:58.885465 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.885465 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:58.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:58.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:58.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:58.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:58.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:58.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:58.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.886858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:58.887523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.887523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:58.887523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:58.887523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:58.887523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:58.887523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:58.887523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:58.887523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:57:58.887523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:58.887523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:58.887523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.887523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:58.887891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:58.888274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.888274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:58.888274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.888274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:58.891355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:58.891849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:58.892376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.892376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:58.892376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:58.892376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.892376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:58.892376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.892376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:58.892376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:58.892376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:58.892376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:58.892715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:58.892715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:58.892715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.892715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:58.892715 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.892715 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:58.892715 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:58.892715 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.892715 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:58.892715 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.892715 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:58.892715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:57:58.893076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.893572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:57:58.893572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:57:58.893572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.893572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:57:58.893572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.893572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:57:58.893572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.893572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:58.893572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:58.893572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:58.893572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:58.893572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:58.893572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:58.893987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:58.893987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:58.893987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.893987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:58.893987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:58.893987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:58.893987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.893987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:58.893987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:58.893987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:58.893987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:58.893987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:58.893987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:58.894377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:58.894377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:58.894377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.894377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:58.894377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:58.894377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.894377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:58.894377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.894377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:58.894377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.894377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:58.894377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.894744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.895290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:57:58.898400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:57:58.898400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:57:58.898400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:57:58.898400 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:57:58.898757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:58.898898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:57:58.899032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.899542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:57:58.899542 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.899542 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:57:58.899542 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:57:58.899542 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.899542 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:57:58.899542 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.899542 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:57:58.900611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.900611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:57:58.900611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:58.900611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:57:58.900611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:57:58.900611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:58.900611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:58.900611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.900611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:58.900611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:58.900611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:57:58.900611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.901248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:57:58.901248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:58.901248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:57:58.901248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:58.901248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:57:58.901248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:57:58.901248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:58.901248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:57:58.901248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.901248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:57:58.901248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:57:58.901248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.901248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:57:58.901727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:57:58.902095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:57:58.902733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:57:58.902733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:57:58.902733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:57:58.902733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:57:58.902733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:57:58.902733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:01.182569 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:01.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:01.183485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:01.184392 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:01.218136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:01.218676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:01.220048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:01.220048 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:01.220048 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:01.220048 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:01.220048 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:01.220048 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:01.220048 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:01.220048 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:01.221201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:01.221201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:01.221201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:01.221201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:01.221201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:01.221201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:01.221201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:01.221201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:01.221201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:01.221201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:01.221201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:01.221201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:01.221802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:01.221802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:01.221802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:01.221802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:01.221802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:01.221802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:01.221802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:01.221802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:01.221802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:01.221802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:01.221802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:01.221802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:01.221802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:01.222486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:01.223261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:01.224254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:01.224254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:01.224254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:01.224254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:01.224254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:01.224254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:03.876725 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:03.877679 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:03.877679 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:03.878019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:03.878279 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:03.878443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.879190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:03.879190 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.879190 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:03.879190 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:03.879190 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.879190 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:03.879190 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.879190 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:03.880336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.880336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:03.880336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:03.880336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:03.880336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:03.880336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:03.880336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:03.880336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.880336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:03.880336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:03.880336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:03.880336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.881102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:03.881102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:03.881102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:03.881102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:03.881102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:03.881102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:03.881102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:03.881102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:03.881102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.881102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:03.881102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:03.881102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.881102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:03.881927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:03.882882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.884148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:03.884148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:03.884148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.884148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:03.884148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.884148 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:03.884148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:03.884148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:03.884696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:03.884696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:58:03.884874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:03.884974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.885488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:03.885488 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.885488 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:03.885488 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:03.885488 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.885488 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:03.885488 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.885488 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:03.886412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:03.887052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:03.887052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:03.887052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.887052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:03.887052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:03.887052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.887052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:03.887052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.887052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:03.887462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.887462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:03.887462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:03.887462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:03.887462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:03.887462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:03.887462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:03.887462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:58:03.887462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:03.887462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:03.887462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.887462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:03.887979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:03.888487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:03.888946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:03.889473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:03.889473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:03.889473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:03.889473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:03.889473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:03.889473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.889473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:03.889473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:03.889473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.889473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:03.889473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.889473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:03.890507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:03.890800 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:03.890904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:03.891033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:03.891394 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:03.891611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.892142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:03.892142 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.892142 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:03.892142 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:03.892142 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.892142 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:03.892142 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.892142 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:03.893225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:03.893643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:03.893643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:03.893643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.893643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:03.893643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:03.893643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.893643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:03.893643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.893643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:03.894368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.894368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:03.894368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:03.894368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:03.894368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:03.894368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:03.894368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:03.894368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:03.894368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:03.894368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.894368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:03.894689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:03.895044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.895044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:03.895044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.895044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:03.895528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:03.896038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:03.896546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.896546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:03.896546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.896546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:03.898454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:03.898744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:03.898840 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:03.899018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:03.899363 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:03.899574 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.900339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:03.900339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.900339 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:03.900339 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:03.900339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.900339 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:03.900339 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.900339 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:03.901368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.901368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:03.901368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:03.901368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:03.901368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:03.901368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:03.901368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:03.901368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.901368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:03.901368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:03.901368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:03.901368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.901852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:03.901852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:03.901852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:03.901852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:03.901852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:03.901852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:03.901852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:03.901852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:03.901852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.901852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:03.901852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:03.901852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.901852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:03.902321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.902321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:03.902465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:03.903051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:03.903841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:03.903841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:03.903841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:06.183195 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:06.184036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:06.184036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:06.185169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:06.220774 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:06.221026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:06.221755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:06.221755 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:06.221755 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:06.221755 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:06.221755 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:06.221755 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:06.221755 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:06.221755 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:06.222555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:06.222555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:06.222555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:06.222555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:06.222555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:06.222555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:06.222555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:06.222555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:06.222555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:06.222555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:06.222555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:06.222555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:06.223336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:06.223336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:06.223336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:06.223336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:06.223336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:06.223336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:06.223336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:06.223336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:06.223336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:06.223336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:06.223336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:06.223336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:06.223336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:06.223996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:06.224774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:06.225711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:06.225711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:06.225711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:06.225711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:06.225711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:06.225711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:08.877336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:08.878534 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:08.878775 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:08.880187 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:08.880585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:08.880890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.882242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:08.882242 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.882242 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:08.882242 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:08.882242 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.882242 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:08.882242 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.882242 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:08.885083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.885083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:08.885083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:08.885083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:08.885083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:08.885083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:08.885083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:08.885083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.885083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:08.885083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:08.885083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:08.885083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:08.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:08.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:08.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:08.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:08.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:08.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:08.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:08.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:08.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:08.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.885538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:08.885979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.885979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:08.887536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:08.888022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.888564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:08.888564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.888564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:08.888564 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:08.888564 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:08.888564 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:08.888564 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:58:08.889159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:08.889581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.891076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:08.891076 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.891076 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:08.891076 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:08.891076 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.891076 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:08.891076 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.891076 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:08.892043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:08.893525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:08.894118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:08.894118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:08.894118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.894118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:08.894118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:08.894118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.894118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:08.894118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.894118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:08.894591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.894591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:08.894591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:08.894591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:08.894591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:08.894591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:08.894591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:08.894591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:58:08.894591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:08.894591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:08.894591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.894591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:08.895042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:08.895499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.895499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:08.895499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.895499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:08.895991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:08.896532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:08.897123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.897123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:08.897123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:08.897123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.897123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:08.897123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.897123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:08.897123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:08.897123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:08.897123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:08.897514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:08.897853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.898673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:08.898673 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.898673 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:08.898673 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:08.898673 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.898673 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:08.898673 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.898673 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:08.899780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:08.900273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:08.900273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:08.900273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.900273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:08.900273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:08.900273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.900273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:08.900273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.900273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:08.900273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:08.901677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.901677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:08.901677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:08.901677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:08.901677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:08.901677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:08.901677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:08.901677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:08.901677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:08.901677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.901677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:08.902061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:08.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:08.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:08.902970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:08.903572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:08.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:08.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:08.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:08.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:08.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:08.904457 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:08.904827 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:08.905064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.905801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:08.905801 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.905801 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:08.905801 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:08.905801 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.905801 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:08.905801 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.905801 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:08.906710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.906710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:08.906710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:08.906710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:08.906710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:08.906710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:08.906710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:08.906710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.906710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:08.906710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:08.906710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:08.906710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.907142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:08.907142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:08.907142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:08.907142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:08.907142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:08.907142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:08.907142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:08.907142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:08.907142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.907142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:08.907142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:08.907142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.907142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:08.907568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:08.908048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:08.908552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:08.908552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:08.908552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:08.908552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:08.908552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:08.908552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:11.183850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:11.184640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:11.184640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:11.185374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:11.196632 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=242 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:58:11.218138 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:11.218545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:11.219390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:11.219390 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:11.219390 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:11.219390 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:11.219390 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:11.219390 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:11.219390 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:11.219390 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:11.220163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:11.220163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:11.220163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:11.220163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:11.220163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:11.220163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:11.220163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:11.220163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:11.220163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:11.220163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:11.220163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:11.220163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:11.220890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:11.220890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:11.220890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:11.220890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:11.220890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:11.220890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:11.220890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:11.220890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:11.220890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:11.220890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:11.220890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:11.220890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:11.220890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:11.221557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:11.222489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:11.223512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:11.223512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:11.223512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:11.223512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:11.223512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:11.223512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:13.878424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:13.879328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:13.879584 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:13.880648 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:13.880899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:13.881053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.881739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:13.881739 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.881739 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:13.881739 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:13.881739 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.881739 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:13.881739 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.881739 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:13.883220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.883220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:13.883220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:13.883220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:13.883220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:13.883220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:13.883220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:13.883220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.883220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:13.883220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:13.883220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:13.883220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.883861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:13.883861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:13.883861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:13.883861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:13.883861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:13.883861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:13.883861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:13.883861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:13.883861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.883861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:13.883861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:13.883861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.883861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:13.884528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.884528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:13.884826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:13.885675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.886593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:13.886593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.886593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:13.888136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:13.888399 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:13.888510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:13.888630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:58:13.889012 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:13.889256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.889781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:13.889781 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.889781 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:13.889781 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:13.889781 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.889781 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:13.889781 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.889781 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:13.890559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:13.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:13.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:13.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:13.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:13.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:13.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.891040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:13.891458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.891458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:13.891458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:13.891458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:13.891458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:13.891458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:13.891458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:13.891458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:58:13.891458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:13.891458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:13.891458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.891458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:13.891827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:13.892231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.892231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:13.892231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.892231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:13.892594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:13.893041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:13.893564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.893564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:13.893564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:13.893564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.893564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:13.893564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.893564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:13.896416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:13.896669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:13.896795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:13.896934 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:13.897344 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:13.897602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.898146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:13.898146 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.898146 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:13.898146 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:13.898146 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.898146 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:13.898146 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.898146 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:13.898997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:13.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:13.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:13.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:13.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:13.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:13.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.899477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:13.901291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.901291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:13.901291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:13.901291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:13.901291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:13.901291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:13.901291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:13.901291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:13.901291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:13.901291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.901291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:13.901700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:13.902120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.902120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:13.902120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.902120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:13.902510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:13.903048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:13.903585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.903585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:13.903585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.903585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:13.903585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:13.903585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:13.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:13.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:13.904121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:13.904410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:13.904578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.905109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:13.905109 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.905109 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:13.905109 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:13.905109 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.905109 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:13.905109 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.905109 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:13.906047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.906047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:13.906047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:13.906047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:13.906047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:13.906047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:13.906047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:13.906047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.906047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:13.906047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:13.906047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:13.906047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.906454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:13.906454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:13.906454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:13.906454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:13.906454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:13.906454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:13.906454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:13.906454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:13.906454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.906454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:13.906454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:13.906454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.906454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:13.906816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.906816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:13.907151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:13.907633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:13.908171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:13.908171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:13.908171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:16.184646 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:16.185967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:16.186148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:16.187497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:16.224713 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:16.225112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:16.226182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:16.226182 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:16.226182 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:16.226182 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:16.226182 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:16.226182 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:16.226182 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:16.226182 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:16.227826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:16.227826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:16.227826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:16.227826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:16.227826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:16.227826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:16.227826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:16.227826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:16.227826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:16.227826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:16.227826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:16.227826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:16.228624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:16.228624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:16.228624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:16.228624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:16.228624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:16.228624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:16.228624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:16.228624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:16.228624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:16.228624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:16.228624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:16.228624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:16.228624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:16.229336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:16.229336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:16.229751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:16.230962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:16.234810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:16.234810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:16.234810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:18.879349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:18.880401 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:18.880401 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:18.881169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:18.881558 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:18.881792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.882806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:18.882806 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.882806 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:18.882806 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:18.882806 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.882806 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:18.882806 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.882806 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:18.884679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.884679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:18.884679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:18.884679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:18.884679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:18.884679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:18.884679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:18.884679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.884679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:18.884679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:18.884679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:18.884679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.885170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:18.885170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:18.885170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:18.885170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:18.885170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:18.885170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:18.885170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:18.885170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:18.885170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.885170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:18.885170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:18.885170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.885170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:18.885688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.885688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:18.887120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:18.887915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.888469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:18.888469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.888469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:18.888561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:18.888915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:18.889047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:18.889217 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:58:18.889618 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:18.889857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.890515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:18.890515 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.890515 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:18.890515 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:18.890515 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.890515 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:18.890515 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.890515 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:18.891441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:18.892040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:18.892040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:18.892040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.892040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:18.892040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:18.892040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.892040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:18.892040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.892040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:18.892480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.892480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:18.892480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:18.892480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:18.892480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:18.892480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:18.892480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:18.892480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:58:18.892480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:18.892480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:18.892480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.892480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:18.892849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:18.893242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.893242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:18.893242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.893242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:18.893732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:18.894478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:18.895248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.895248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:18.895248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:18.895248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.895248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:18.895248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.895248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:18.896902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:18.898036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:18.898166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:18.898379 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:18.899032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:18.899391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.900227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:18.900227 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.900227 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:18.900227 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:18.900227 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.900227 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:18.900227 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.900227 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:18.901440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:18.902055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:18.902055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:18.902055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.902055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:18.902055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:18.902055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.902055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:18.902055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.902055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:18.902393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.902393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:18.902393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:18.902393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:18.902393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:18.902393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:18.902393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:18.902393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:18.902393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:18.902393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.902393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:18.902717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:18.903097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.903097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:18.903097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.903097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:18.903803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:18.904366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:18.905000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.905000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:18.905000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.905000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:18.905000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:18.905000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:18.905000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:18.905000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:18.905615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:18.906155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:18.906506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.907572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:18.907572 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.907572 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:18.907572 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:18.907572 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.907572 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:18.907572 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.907572 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:18.908493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.908493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:18.908493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:18.908493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:18.908493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:18.908493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:18.908493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:18.908493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.908493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:18.908493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:18.908493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:18.908493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.908866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:18.908866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:18.908866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:18.908866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:18.908866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:18.908866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:18.908866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:18.908866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:18.908866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.908866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:18.908866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:18.908866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.908866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:18.909264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.909264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:18.909711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:18.910330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:18.911080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:18.911080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:18.911080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:21.186165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:21.186945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:21.186945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:21.188189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:21.222726 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:21.222968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:21.223488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:21.223488 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:21.223488 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:21.223488 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:21.223488 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:21.223488 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:21.223488 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:21.223488 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:21.224541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:21.224541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:21.224541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:21.224541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:21.224541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:21.224541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:21.224541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:21.224541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:21.224541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:21.224541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:21.224541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:21.224541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:21.224946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:21.224946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:21.224946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:21.224946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:21.224946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:21.224946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:21.224946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:21.224946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:21.224946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:21.224946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:21.224946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:21.224946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:21.224946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:21.225347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:21.225347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:21.225749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:21.226251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:21.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:21.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:21.226839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:23.879951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:23.880646 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:23.880646 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:23.881542 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:23.881845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:23.882191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.882901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:23.882901 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.882901 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:23.882901 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:23.882901 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.882901 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:23.882901 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.882901 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:23.883925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.883925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:23.883925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:23.883925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:23.883925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:23.883925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:23.883925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:23.883925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.883925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:23.883925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:23.883925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:23.883925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.884542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:23.884542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:23.884542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:23.884542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:23.884542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:23.884542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:23.884542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:23.884542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:23.884542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.884542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:23.884542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:23.884542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.884542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:23.885071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:23.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.886591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:23.886591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:23.886591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.886591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:23.886591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.886591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:23.889039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:23.889321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:23.889428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:23.889508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:58:23.889970 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:23.890269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.890934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:23.890934 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.890934 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:23.890934 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:23.890934 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.890934 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:23.890934 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.890934 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:23.891666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:23.892312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:23.892312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:23.892312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.892312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:23.892312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:23.892312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.892312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:23.892312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.892312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:23.892614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.892614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:23.892614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:23.892614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:23.892614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:23.892614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:23.892614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:23.892614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:58:23.892614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:23.892614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:23.892614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.892614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:23.892994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:23.893415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.893415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:23.893415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.893415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:23.893739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:23.894266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:23.894794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.894794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:23.894794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:23.894794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.894794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:23.894794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.894794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:23.897450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:23.897723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:23.897818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:23.897931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:23.898636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:23.899010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.899696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:23.899696 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.899696 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:23.899696 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:23.899696 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.899696 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:23.899696 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.899696 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:23.900412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:23.900876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:23.900876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:23.900876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.900876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:23.900876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:23.900876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.900876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:23.900876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.900876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:23.901254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.901254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:23.901254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:23.901254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:23.901254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:23.901254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:23.901254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:23.901254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:23.901254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:23.901254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.901254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:23.901608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:23.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:23.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:23.902300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:23.902832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:23.903417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.903417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:23.903417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.903417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:23.904765 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:23.905035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:23.905132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:23.905284 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:23.905617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:23.905845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:23.905995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.906515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:23.906515 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.906515 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:23.906515 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:23.906515 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.906515 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:23.906515 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.906515 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:23.907502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.907502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:23.907502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:23.907502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:23.907502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:23.907502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:23.907502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:23.907502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.907502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:23.907502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:23.907502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:23.907502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.907896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:23.907896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:23.907896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:23.907896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:23.907896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:23.907896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:23.907896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:23.907896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:23.907896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.907896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:23.907896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:23.907896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.907896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:23.908277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.908277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:23.908485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:23.908971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:23.909528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:23.909528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:23.909528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:26.186872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:26.187695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:26.187760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:26.189168 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:26.224407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:26.224602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:26.225185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:26.225185 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:26.225185 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:26.225185 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:26.225185 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:26.225185 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:26.225185 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:26.225185 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:26.226342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:26.226342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:26.226342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:26.226342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:26.226342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:26.226342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:26.226342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:26.226342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:26.226342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:26.226342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:26.226342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:26.226342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:26.226905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:26.226905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:26.226905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:26.226905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:26.226905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:26.226905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:26.226905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:26.226905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:26.226905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:26.226905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:26.226905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:26.226905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:26.226905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:26.227467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:26.228055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:26.228808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:26.228808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:26.228808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:26.228808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:26.228808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:26.228808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:28.880663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:28.881643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:28.881643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:28.882644 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:28.883103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:28.883772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.884151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:28.884151 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.884151 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:28.884151 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:28.884151 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.884151 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:28.884151 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.884151 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:28.885493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.885493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:28.885493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:28.885493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:28.885493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:28.885493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:28.885493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:28.885493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.885493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:28.885493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:28.885493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:28.885493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.886283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:28.886283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:28.886283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:28.886283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:28.886283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:28.886283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:28.886283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:28.886283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:28.886283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.886283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:28.886283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:28.886283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.886283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:28.887440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:28.887913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.888693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:28.888693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:28.888693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.888693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:28.888693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.888693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:28.889412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:28.889712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:28.889845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:28.889974 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:58:28.890482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:28.890793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.891629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:28.891629 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.891629 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:28.891629 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:28.891629 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.891629 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:28.891629 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.891629 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:28.892505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:28.893492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:28.893961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:28.893961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:28.893961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:58:28.893961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:28.893961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:28.893961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.893961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:28.893961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:28.893961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:28.893961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.893961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:28.893961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:28.893961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:28.894388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:28.894388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:28.894388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:28.894388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:28.894388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:28.894388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.894388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:28.894388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:28.894388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.894388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:28.894388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.894388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:28.894796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:28.895500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:28.896295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.896295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:28.896295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:28.896295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.896295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:28.896295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.896295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:28.897776 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:28.898060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:28.898256 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:28.898459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:28.899091 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:28.899518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.900459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:28.900459 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.900459 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:28.900459 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:28.900459 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.900459 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:28.900459 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.900459 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:28.902019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:28.902568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:28.902568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:28.902568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.902568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:28.902568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:28.902568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.902568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:28.902568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.902568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:28.903864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.903864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:28.903864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:28.903864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:28.903864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:28.903864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:28.903864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:28.903864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:28.903864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:28.903864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.903864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:28.904500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:28.905087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.905087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:28.905087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.905087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:28.905303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:28.905977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:28.906781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.906781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:28.906781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.906781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:28.906781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:28.906781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:28.906781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:28.906781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:28.907132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:28.907132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:28.907323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.907983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:28.907983 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.907983 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:28.907983 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:28.907983 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.907983 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:28.907983 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.907983 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:28.908870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.908870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:28.908870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:28.908870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:28.908870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:28.908870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:28.908870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:28.908870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.908870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:28.908870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:28.908870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:28.908870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.909426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:28.909426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:28.909426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:28.909426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:28.909426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:28.909426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:28.909426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:28.909426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:28.909426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.909426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:28.909426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:28.909426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.909426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:28.909814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.909814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:28.909884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:28.910358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:28.910920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:28.910920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:28.910920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:31.187523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:31.188366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:31.188366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:31.189547 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:31.220649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:31.221283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:31.221894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:31.221894 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:31.221894 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:31.221894 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:31.221894 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:31.221894 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:31.221894 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:31.221894 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:31.222657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:31.222657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:31.222657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:31.222657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:31.222657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:31.222657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:31.222657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:31.222657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:31.222657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:31.222657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:31.222657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:31.222657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:31.223242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:31.223242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:31.223242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:31.223242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:31.223242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:31.223242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:31.223242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:31.223242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:31.223242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:31.223242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:31.223242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:31.223242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:31.223242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:31.223912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:31.224629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:31.225103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:31.225103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:31.225103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:31.225103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:31.225103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:31.225103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:33.881412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:33.882323 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:33.882323 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:33.882798 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:33.883033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:33.883195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.883934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:33.883934 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.883934 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:33.883934 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:33.883934 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.883934 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:33.883934 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.883934 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:33.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:33.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:33.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:33.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:33.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:33.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:33.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:33.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:33.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:33.885524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.886193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:33.886193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:33.886193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:33.886193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:33.886193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:33.886193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:33.886193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:33.886193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:33.886193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.886193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:33.886193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:33.886193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.886193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:33.886875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.886875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:33.887514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:33.888366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.889377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:33.889377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.889377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:33.890023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:33.890361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:33.890496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:33.890622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:58:33.891048 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:33.891363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.892038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:33.892038 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.892038 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:33.892038 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:33.892038 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.892038 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:33.892038 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.892038 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:33.892955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:33.893582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:33.893582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:33.893582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.893582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:33.893582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:33.893582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.893582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:33.893582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.893582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:33.894226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.894226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:33.894226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:33.894226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:33.894226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:33.894226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:33.894226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:33.894226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:58:33.894226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:33.894226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:33.894226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.894226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:33.894814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:33.895362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.895362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:33.895362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.895362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:33.895487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:33.895964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:33.896512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.896512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:33.896512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:33.896512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.896512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:33.896512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.896512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:33.898230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:33.898801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:33.898918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:33.899043 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:33.899502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:33.899845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.900520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:33.900520 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.900520 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:33.900520 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:33.900520 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.900520 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:33.900520 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.900520 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:33.901713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:33.902357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:33.902357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:33.902357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.902357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:33.902357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:33.902357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.902357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:33.902357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.902357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:33.903360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.903360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:33.903360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:33.903360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:33.903360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:33.903360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:33.903360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:33.903360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:33.903360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:33.903360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.903360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:33.903800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:33.904237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.904237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:33.904237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.904237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:33.904605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:33.905247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:33.906004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.906004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:33.906004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.906004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:33.906177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:33.906356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:33.906471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:33.906638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:33.907024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:33.907381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:33.907586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.908215 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:33.908215 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.908215 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:33.908215 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:33.908215 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.908215 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:33.908215 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.908215 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:33.909075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.909075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:33.909075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:33.909075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:33.909075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:33.909075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:33.909075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:33.909075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.909075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:33.909075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:33.909075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:33.909075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.909512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:33.909512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:33.909512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:33.909512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:33.909512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:33.909512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:33.909512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:33.909512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:33.909512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.909512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:33.909512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:33.909512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.909512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:33.909868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.909868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:33.910122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:33.910602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:33.911267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:33.911267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:33.911267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:36.188185 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:36.188802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:36.189126 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:36.190159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:36.229382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:36.229766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:36.231309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:36.231309 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:36.231309 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:36.231309 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:36.231309 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:36.231309 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:36.231309 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:36.231309 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:36.231728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:36.231728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:36.231728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:36.231728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:36.231728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:36.231728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:36.231728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:36.231728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:36.231728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:36.231728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:36.231728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:36.231728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:36.232161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:36.232161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:36.232161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:36.232161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:36.232161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:36.232161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:36.232161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:36.232161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:36.232161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:36.232161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:36.232161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:36.232161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:36.232161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:36.233390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:36.234136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:36.234706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:36.234706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:36.234706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:36.234706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:36.234706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:36.234706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:38.882012 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:38.883037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:38.883037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:38.884622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:38.884965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:38.885259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.886295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:38.886295 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.886295 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:38.886295 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:38.886295 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.886295 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:38.886295 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.886295 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:38.887309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.887309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:38.887309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:38.887309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:38.887309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:38.887309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:38.887309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:38.887309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.887309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:38.887309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:38.887309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:38.887309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.887974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:38.887974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:38.887974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:38.887974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:38.887974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:38.887974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:38.887974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:38.887974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:38.887974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.887974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:38.887974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:38.887974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.887974 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:38.888494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.888494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:38.889872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:38.890584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.891387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:38.891387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.891387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:38.891387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:38.891387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:38.891387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:38.891387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:58:38.891740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:38.891785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.892519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:38.892519 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.892519 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:38.892519 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:38.892519 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.892519 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:38.892519 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.892519 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:38.893424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:38.893933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:38.893933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:38.893933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.893933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:38.893933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:38.893933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.893933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:38.893933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.893933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:38.894305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.894305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:38.894305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:38.894305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:38.894305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:38.894305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:38.894305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:38.894305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:58:38.894305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:38.894305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:38.894305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.894305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:38.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:38.895170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.895170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:38.895170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.895170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:38.895526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:38.895992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:38.896562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.896562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:38.896562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:38.896562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.896562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:38.896562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.896562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:38.898816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:38.899074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:38.899178 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:38.899361 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:38.899745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:38.900000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.900616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:38.900616 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.900616 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:38.900616 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:38.900616 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.900616 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:38.900616 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.900616 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:38.901445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:38.901860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:38.901860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:38.901860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.901860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:38.901860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:38.901860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.901860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:38.901860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.901860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:38.902372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.902372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:38.902372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:38.902372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:38.902372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:38.902372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:38.902372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:38.902372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:38.902372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:38.902372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.902372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:38.902734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:38.903173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.903173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:38.903173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.903173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:38.903514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:38.904028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:38.904592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.904592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:38.904592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.904592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:38.906407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:38.906649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:38.906770 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:38.906940 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:38.907316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:38.907545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:38.907699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.908271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:38.908271 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.908271 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:38.908271 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:38.908271 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.908271 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:38.908271 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.908271 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:38.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:38.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:38.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:38.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:38.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:38.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:38.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:38.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:38.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:38.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.909616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:38.909616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:38.909616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:38.909616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:38.909616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:38.909616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:38.909616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:38.909616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:38.909616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.909616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:38.909616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:38.909616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.909616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:38.909995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.909995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:38.910161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:38.910651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:38.911198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:38.911198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:38.911198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:41.188938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:41.189748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:41.189748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:41.190877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:41.223703 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:41.223985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:41.224681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:41.224681 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:41.224681 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:41.224681 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:41.224681 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:41.224681 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:41.224681 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:41.224681 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:41.225413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:41.225413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:41.225413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:41.225413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:41.225413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:41.225413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:41.225413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:41.225413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:41.225413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:41.225413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:41.225413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:41.225413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:41.225896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:41.225896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:41.225896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:41.225896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:41.225896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:41.225896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:41.225896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:41.225896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:41.225896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:41.225896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:41.225896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:41.225896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:41.225896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:41.226290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:41.226870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:41.227665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:41.227665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:41.227665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:41.227665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:41.227665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:41.227665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:43.882609 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:43.883196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:43.883269 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:43.884203 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:43.884453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:43.884613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.885234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:43.885234 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.885234 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:43.885234 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:43.885234 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.885234 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:43.885234 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.885234 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:43.886271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.886271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:43.886271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:43.886271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:43.886271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:43.886271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:43.886271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:43.886271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.886271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:43.886271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:43.886271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:43.886271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.886675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:43.886675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:43.886675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:43.886675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:43.886675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:43.886675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:43.886675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:43.886675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:43.886675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.886675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:43.886675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:43.886675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.886675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:43.887079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.887079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:43.887318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:43.887817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.888441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:43.888441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.888441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:43.891142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:43.891493 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:43.891606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:43.891721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:58:43.892070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:43.892314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.892916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:43.892916 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.892916 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:43.892916 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:43.892916 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.892916 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:43.892916 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.892916 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:43.893680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:43.894153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:43.894153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:43.894153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.894153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:43.894153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:43.894153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.894153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:43.894153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.894153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:43.894520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.894520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:43.894520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:43.894520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:43.894520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:43.894520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:43.894520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:43.894520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:58:43.894520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:43.894520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:43.894520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.894520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:43.894869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:43.895251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.895251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:43.895251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.895251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:43.895903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:43.896347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:43.896832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.896832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:43.896832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:43.896832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.896832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:43.896832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.896832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:43.899459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:43.899672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:43.900349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:43.900349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:43.900349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:43.900597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.901060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:43.901060 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.901060 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:43.901060 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:43.901060 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.901060 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:43.901060 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.901060 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:43.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:43.902517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:43.902517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:43.902517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.902517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:43.902517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:43.902517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.902517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:43.902517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.902517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:43.902932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.902932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:43.902932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:43.902932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:43.902932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:43.902932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:43.902932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:43.902932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:43.902932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:43.902932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.902932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:43.903450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:43.904040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:43.904485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:43.905044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:43.905044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:43.905044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:43.905044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:43.905044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.905044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:43.905044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:43.905044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.905044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:43.905044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.905044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:43.906755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:43.906997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:43.907138 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:43.907305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:43.907630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:43.907886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:43.908114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.908994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:43.908994 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.908994 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:43.908994 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:43.908994 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.908994 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:43.908994 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.908994 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:43.910521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.910521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:43.910521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:43.910521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:43.910521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:43.910521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:43.910521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:43.910521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.910521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:43.910521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:43.910521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:43.910521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.910907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:43.910907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:43.910907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:43.910907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:43.910907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:43.910907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:43.910907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:43.910907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:43.910907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.910907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:43.910907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:43.910907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.910907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:43.911396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:43.911918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:43.912455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:43.912455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:43.912455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:43.912455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:43.912455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:43.912455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:46.189844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:46.190639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:46.190639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:46.191249 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:46.224852 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:46.225135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:46.225868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:46.225868 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:46.225868 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:46.225868 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:46.225868 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:46.225868 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:46.225868 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:46.225868 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:46.226549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:46.226549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:46.226549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:46.226549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:46.226549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:46.226549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:46.226549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:46.226549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:46.226549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:46.226549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:46.226549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:46.226549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:46.227118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:46.227118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:46.227118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:46.227118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:46.227118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:46.227118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:46.227118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:46.227118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:46.227118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:46.227118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:46.227118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:46.227118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:46.227118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:46.227717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:46.228419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:46.229331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:46.229331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:46.229331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:46.229331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:46.229331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:46.229331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:48.883395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:48.884111 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:48.884390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:48.885544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:48.885762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:48.886016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.886915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:48.886915 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.886915 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:48.886915 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:48.886915 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.886915 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:48.886915 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.886915 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:48.887909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.887909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:48.887909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:48.887909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:48.887909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:48.887909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:48.887909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:48.887909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.887909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:48.887909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:48.887909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:48.887909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.888298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:48.888298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:48.888298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:48.888298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:48.888298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:48.888298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:48.888298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:48.888298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:48.888298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.888298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:48.888298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:48.888298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.888298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:48.888621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.888621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:48.888998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:48.889456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.889960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:48.889960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.889960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:48.891557 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:48.891793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:48.891904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:48.892081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:58:48.892902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:48.893308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.893971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:48.893971 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.893971 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:48.893971 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:48.893971 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.893971 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:48.893971 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.893971 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:48.894697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:48.895095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:48.895095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:48.895095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.895095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:48.895095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:48.895095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.895095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:48.895095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.895095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:48.895534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.895534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:48.895534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:48.895534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:48.895534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:48.895534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:48.895534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:48.895534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:58:48.895534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:48.895534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:48.895534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.895534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:48.895864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:48.896223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.896223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:48.896223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.896223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:48.896587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:48.897002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:48.897460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.897460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:48.897460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:48.897460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.897460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:48.897460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.897460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:48.899907 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:48.900736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:48.900895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:48.901412 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:48.901661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:48.901893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.902532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:48.902532 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.902532 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:48.902532 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:48.902532 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.902532 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:48.902532 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.902532 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:48.903741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:48.904216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:48.904216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:48.904216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.904216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:48.904216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:48.904216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.904216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:48.904216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.904216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:48.905271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.905271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:48.905271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:48.905271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:48.905271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:48.905271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:48.905271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:48.905271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:48.905271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:48.905271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.905271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:48.905632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:48.906021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.906021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:48.906021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.906021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:48.908074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:48.908655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:48.909349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.909349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:48.909349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.909349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:48.909349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:48.909349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:48.909349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:48.909349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:48.910625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:48.910985 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:48.911132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.915596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:48.915596 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.915596 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:48.915596 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:48.915596 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.915596 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:48.915596 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.915596 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:48.916263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.916263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:48.916263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:48.916263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:48.916263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:48.916263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:48.916263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:48.916263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.916263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:48.916263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:48.916263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:48.916263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.916657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:48.916657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:48.916657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:48.916657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:48.916657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:48.916657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:48.916657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:48.916657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:48.916657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.916657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:48.916657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:48.916657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.916657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:48.917055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:48.917542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:48.918084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:48.918084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:48.918084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:48.918084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:48.918084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:48.918084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:51.190364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:51.191115 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:51.191115 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:51.192273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:51.225233 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:51.225614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:51.226416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:51.226416 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:51.226416 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:51.226416 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:51.226416 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:51.226416 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:51.226416 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:51.226416 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:51.227499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:51.227499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:51.227499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:51.227499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:51.227499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:51.227499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:51.227499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:51.227499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:51.227499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:51.227499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:51.227499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:51.227499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:51.228107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:51.228107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:51.228107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:51.228107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:51.228107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:51.228107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:51.228107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:51.228107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:51.228107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:51.228107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:51.228107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:51.228107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:51.228107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:51.228737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:51.229466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:51.230350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:51.230350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:51.230350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:51.230350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:51.230350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:51.230350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:53.884342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:53.885052 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:53.885103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:53.885962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:53.886162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:53.886426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.887359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:53.887359 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.887359 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:53.887359 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:53.887359 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.887359 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:53.887359 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.887359 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:53.888827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.888827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:53.888827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:53.888827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:53.888827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:53.888827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:53.888827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:53.888827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.888827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:53.888827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:53.888827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:53.888827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.889482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:53.889482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:53.889482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:53.889482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:53.889482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:53.889482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:53.889482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:53.889482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:53.889482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.889482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:53.889482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:53.889482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.889482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:53.890117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.890117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:53.890479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:53.891261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.892263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:53.892263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.892263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:53.892263 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:53.892414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:53.892546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:53.892662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:58:53.893117 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:53.893475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.894117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:53.894117 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.894117 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:53.894117 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:53.894117 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.894117 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:53.894117 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.894117 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:53.894855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:53.895347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:53.895347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:53.895347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.895347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:53.895347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:53.895347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.895347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:53.895347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.895347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:53.895793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.895793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:53.895793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:53.895793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:53.895793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:53.895793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:53.895793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:53.895793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:58:53.895793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:53.895793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:53.895793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.895793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:53.896188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:53.896593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.896593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:53.896593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.896593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:53.896841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:53.897305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:53.897950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.897950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:53.897950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:53.897950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.897950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:53.897950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.897950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:53.900608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:53.900911 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:53.901020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:53.901138 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:53.901469 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:53.901708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.902139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:53.902139 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.902139 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:53.902139 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:53.902139 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.902139 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:53.902139 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.902139 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:53.902972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:53.903633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:53.903633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:53.903633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.903633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:53.903633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:53.903633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.903633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:53.903633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.903633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:53.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:53.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:53.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:53.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:53.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:53.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:53.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:53.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:53.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.904151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:53.904521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:53.904907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.904907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:53.904907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.904907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:53.905042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:53.905651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:53.906260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.906260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:53.906260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.906260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:53.908925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:53.909251 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:53.909531 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:53.909531 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:53.909909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:53.910303 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:53.910590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.911192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:53.911192 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.911192 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:53.911192 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:53.911192 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.911192 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:53.911192 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.911192 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:53.912089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.912089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:53.912089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:53.912089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:53.912089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:53.912089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:53.912089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:53.912089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.912089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:53.912089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:53.912089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:53.912089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.912682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:53.912682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:53.912682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:53.912682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:53.912682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:53.912682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:53.912682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:53.912682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:53.912682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.912682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:53.912682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:53.912682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.912682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:53.913247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:53.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:53.914727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:53.914727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:53.914727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:53.914727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:53.914727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:53.914727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:56.190931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:56.191508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:56.191508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:56.192125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:56.224648 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:56.224901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:56.225605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:56.225605 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:56.225605 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:56.225605 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:56.225605 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:56.225605 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:56.225605 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:56.225605 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:56.226384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:56.226384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:56.226384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:56.226384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:56.226384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:56.226384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:56.226384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:56.226384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:56.226384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:56.226384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:56.226384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:56.226384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:56.226670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:56.226670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:56.226670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:56.226670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:56.226670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:56.226670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:56.226670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:56.226670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:56.226670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:56.226670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:56.226670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:56.226670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:56.226670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:56.226923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:56.226923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:56.227320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:56.227680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:56.228137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:56.228137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:56.228137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:58.884947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:58.888264 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:58.888264 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:58.888659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:58.888923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:58.889056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:58.892707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:58.893486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:58.894015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:58.894543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:58.895364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:58.895364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:58.895364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:58.895364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.895364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:58.895364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:58.895364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.895364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:58.895364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.895364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:58.895364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:58.896650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:58.896650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:58.896650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:58:58.899755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:58.900028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.901035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:58.901035 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.901035 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:58.901035 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:58.901035 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.901035 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:58.901035 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.901035 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:58.901739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:58.902135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:58.902135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:58.902135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.902135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:58.902135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:58.902135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.902135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:58.902135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.902135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:58.902736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:58.904823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.904823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:58.904823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:58.904823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:58.904823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:58.904823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:58.904823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:58.904823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:58:58.904823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:58.904823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:58.904823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.904823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:58.905151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:58.905458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:58.905748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:58.909520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:58.910045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:58.910045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:58.910045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.910045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:58.910045 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.910045 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:58.910045 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:58.910045 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.910045 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:58.910045 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.910045 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:58.910045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:58:58.910434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.911032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:58:58.911032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:58:58.911032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.911032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:58:58.911032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.911032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:58:58.911032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.911032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:58.911032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:58.911032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:58.911032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:58.911032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:58.911032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:58.911551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:58.911551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:58.911551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.911551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:58.911551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:58.911551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:58.911551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.911551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:58.911551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:58.911551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:58.911551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:58.911551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:58.911551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:58.912042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:58.912042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:58.912042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.912042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:58.912042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:58.912042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.912042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:58.912042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.912042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:58.912042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.912042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:58.912042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.912428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.914001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:58:58.914680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:58:58.914680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:58:58.914680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:58:58.914680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:58:58.914680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:58.914680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:58:58.914933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.915541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:58:58.915541 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.915541 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:58:58.915541 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:58:58.915541 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.915541 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:58:58.915541 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.915541 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:58:58.918279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.918279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:58:58.918279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:58.918279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:58:58.918279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:58:58.918279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:58.918279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:58.918279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.918279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:58.918279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:58.918279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:58:58.918279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:58:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:58:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:58:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:58:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:58:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:58:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:58:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:58:58.919132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:58:58.919727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:58:58.920703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:58:58.920703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:58:58.920703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:58:58.920703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:58:58.920703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:58:58.920703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:01.191766 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:01.192706 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:01.192706 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:01.193426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:01.226374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:01.226611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:01.227423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:01.227423 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:01.227423 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:01.227423 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:01.227423 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:01.227423 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:01.227423 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:01.227423 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:01.228110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:01.228110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:01.228110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:01.228110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:01.228110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:01.228110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:01.228110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:01.228110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:01.228110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:01.228110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:01.228110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:01.228110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:01.228482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:01.228482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:01.228482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:01.228482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:01.228482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:01.228482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:01.228482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:01.228482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:01.228482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:01.228482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:01.228482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:01.228482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:01.228482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:01.228775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:01.228775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:01.229013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:01.229747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:01.230773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:01.230773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:01.230773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:03.886947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:03.887614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:03.887614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:03.888116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:03.888377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:03.888538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.889323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:03.889323 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.889323 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:03.889323 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:03.889323 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.889323 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:03.889323 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.889323 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:03.890055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.890055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:03.890055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:03.890055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:03.890055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:03.890055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:03.890055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:03.890055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.890055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:03.890055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:03.890055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:03.890055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.890348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:03.890348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:03.890348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:03.890348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:03.890348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:03.890348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:03.890348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:03.890348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:03.890348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.890348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:03.890348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:03.890348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.890348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:03.890649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.890649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:03.890922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:03.891825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.892639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:03.892639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.892639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:03.896380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:03.897267 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:03.897384 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:03.897444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:59:03.897986 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:03.898219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.898995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:03.898995 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.898995 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:03.898995 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:03.898995 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.898995 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:03.898995 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.898995 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:03.899757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:03.900253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:03.900253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:03.900253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.900253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:03.900253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:03.900253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.900253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:03.900253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.900253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:03.900518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.900518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:03.900518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:03.900518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:03.900518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:03.900518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:03.900518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:03.900518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:59:03.900518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:03.900518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:03.900518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.900518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:03.900867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:03.901244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.901244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:03.901244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.901244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:03.901880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:03.903188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:03.904883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.904883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:03.904883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:03.904883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.904883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:03.904883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.904883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:03.904883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:03.904883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:03.905453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:03.905453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:03.905686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:03.905950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.906617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:03.906617 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.906617 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:03.906617 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:03.906617 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.906617 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:03.906617 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.906617 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:03.907192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:03.907652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:03.907652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:03.907652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.907652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:03.907652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:03.907652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.907652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:03.907652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.907652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:03.908130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.908130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:03.908130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:03.908130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:03.908130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:03.908130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:03.908130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:03.908130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:03.908130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:03.908130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.908130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:03.908949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:03.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:03.910066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:03.911497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:03.911497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:03.911497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:03.911497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:03.911497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.911497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:03.911497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:03.911497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.911497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:03.911497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.911497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:03.913351 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:03.914524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:03.914643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:03.914809 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:03.915328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:03.915643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:03.915789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.916388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:03.916388 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.916388 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:03.916388 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:03.916388 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.916388 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:03.916388 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.916388 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:03.917242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.917242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:03.917242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:03.917242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:03.917242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:03.917242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:03.917242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:03.917242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.917242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:03.917242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:03.917242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:03.917242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.917652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:03.917652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:03.917652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:03.917652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:03.917652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:03.917652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:03.917652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:03.917652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:03.917652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.917652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:03.917652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:03.917652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.917652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:03.917957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.917957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:03.918431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:03.919539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:03.920193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:03.920193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:03.920193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:06.193254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:06.193254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:06.193254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:06.194049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:06.224480 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:06.225019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:06.225936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:06.225936 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:06.225936 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:06.225936 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:06.225936 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:06.225936 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:06.225936 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:06.225936 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:06.226757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:06.226757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:06.226757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:06.226757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:06.226757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:06.226757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:06.226757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:06.226757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:06.226757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:06.226757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:06.226757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:06.226757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:06.227269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:06.227269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:06.227269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:06.227269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:06.227269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:06.227269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:06.227269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:06.227269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:06.227269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:06.227269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:06.227269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:06.227269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:06.227269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:06.227853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:06.228418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:06.229119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:06.229119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:06.229119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:06.229119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:06.229119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:06.229119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:08.887461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:08.887904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:08.887977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:08.888967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:08.889222 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:08.889588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.890384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:08.890384 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.890384 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:08.890384 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:08.890384 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.890384 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:08.890384 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.890384 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:08.891402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.891402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:08.891402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:08.891402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:08.891402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:08.891402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:08.891402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:08.891402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.891402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:08.891402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:08.891402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:08.891402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.891676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:08.891676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:08.891676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:08.891676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:08.891676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:08.891676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:08.891676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:08.891676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:08.891676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.891676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:08.891676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:08.891676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.891676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:08.891906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.891906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:08.892438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:08.892833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.893178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:08.893178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.893178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:08.897501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:08.897817 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:08.897896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:08.897993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:59:08.898692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:08.898909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.899595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:08.899595 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.899595 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:08.899595 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:08.899595 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.899595 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:08.899595 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.899595 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:08.900270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:08.900602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:08.900602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:08.900602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.900602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:08.900602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:08.900602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.900602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:08.900602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.900602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:08.900934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.900934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:08.900934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:08.900934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:08.900934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:08.900934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:08.900934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:08.900934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:59:08.900934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:08.900934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:08.900934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.900934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:08.901504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:08.902175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:08.902543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:08.902929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:08.902929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:08.902929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:08.902929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:08.902929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:08.902929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.902929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:08.902929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:08.902929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.902929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:08.902929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.902929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:08.905306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:08.905306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:08.905306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:08.905306 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:08.905866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:08.906138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.906856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:08.906856 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.906856 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:08.906856 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:08.906856 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.906856 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:08.906856 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.906856 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:08.907733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:08.908039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:08.908039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:08.908039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.908039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:08.908039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:08.908039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.908039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:08.908039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.908039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:08.908743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.908743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:08.908743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:08.908743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:08.908743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:08.908743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:08.908743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:08.908743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:08.908743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:08.908743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.908743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:08.908985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:08.909245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.909245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:08.909245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.909245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:08.909901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:08.910335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:08.910709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.910709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:08.910709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.910709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:08.914589 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:08.915029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:08.915150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:08.915353 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:08.916147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:08.916647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:08.916795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.917539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:08.917539 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.917539 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:08.917539 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:08.917539 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.917539 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:08.917539 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.917539 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:08.918502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.918502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:08.918502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:08.918502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:08.918502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:08.918502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:08.918502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:08.918502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.918502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:08.918502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:08.918502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:08.918502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.919054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:08.919054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:08.919054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:08.919054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:08.919054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:08.919054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:08.919054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:08.919054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:08.919054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.919054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:08.919054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:08.919054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.919054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:08.919326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.919326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:08.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:08.919890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:08.920261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:08.920261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:08.920261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:11.192959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:11.193535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:11.193535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:11.194391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:11.203634 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=239 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 15:59:11.239964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:11.240252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:11.241427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:11.241427 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:11.241427 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:11.241427 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:11.241427 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:11.241427 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:11.241427 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:11.241427 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:11.242340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:11.242340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:11.242340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:11.242340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:11.242340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:11.242340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:11.242340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:11.242340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:11.242340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:11.242340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:11.242340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:11.242340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:11.242930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:11.242930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:11.242930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:11.242930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:11.242930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:11.242930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:11.242930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:11.242930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:11.242930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:11.242930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:11.242930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:11.242930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:11.242930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:11.243470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:11.244091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:11.244867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:11.244867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:11.244867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:11.244867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:11.244867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:11.244867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:13.888014 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:13.889895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:13.889895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:13.890015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:13.890854 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:13.890854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.891524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:13.891524 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.891524 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:13.891524 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:13.891524 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.891524 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:13.891524 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.891524 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:13.893195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.893195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:13.893195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:13.893195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:13.893195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:13.893195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:13.893195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:13.893195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.893195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:13.893195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:13.893195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:13.893195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.894003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:13.894003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:13.894003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:13.894003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:13.894003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:13.894003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:13.894003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:13.894003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:13.894003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.894003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:13.894003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:13.894003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.894003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:13.894793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.894793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:13.896018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:13.896938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.897558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:13.897558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.897558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:13.899445 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:13.899445 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:13.899445 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:13.899445 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:59:13.899445 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:13.899445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.900124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:13.900124 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.900124 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:13.900124 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:13.900124 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.900124 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:13.900124 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.900124 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:13.900914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:13.901782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:13.901782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:13.901782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.901782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:13.901782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:13.901782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.901782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:13.901782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.901782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:13.903200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.903200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:13.903200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:13.903200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:13.903200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:13.903200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:13.903200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:13.903200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:59:13.903200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:13.903200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:13.903200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.903200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:13.904312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:13.904690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:13.905075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:13.905613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:13.906756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:13.906756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:13.906756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:13.906756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.907292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:13.907292 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.907292 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:13.907292 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:13.907292 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.907292 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:13.907292 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.907292 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:13.907805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:13.908079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:13.908079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:13.908079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.908079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:13.908079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:13.908079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.908079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:13.908079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.908079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:13.908489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.908489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:13.908489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:13.908489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:13.908489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:13.908489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:13.908489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:13.908489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:13.908489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:13.908489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.908489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:13.909237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:13.909476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:13.909746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:13.910091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:13.910091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:13.910091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:13.910091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:13.910091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.910091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:13.910091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:13.910091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.910091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:13.910091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.910091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:13.914787 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:13.915054 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:13.915150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:13.915275 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:13.915651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:13.915869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:13.915994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.916496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:13.916496 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.916496 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:13.916496 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:13.916496 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.916496 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:13.916496 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.916496 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:13.917313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.917313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:13.917313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:13.917313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:13.917313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:13.917313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:13.917313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:13.917313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.917313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:13.917313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:13.917313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:13.917313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.918197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:13.918197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:13.918197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:13.918197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:13.918197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:13.918197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:13.918197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:13.918197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:13.918197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.918197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:13.918197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:13.918197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.918197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:13.918588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:13.919437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:13.920761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:13.920761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:13.920761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:13.920761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:13.920761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:13.920761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:16.195662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:16.196505 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:16.196505 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:16.196919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:16.228605 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:16.229010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:16.229879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:16.229879 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:16.229879 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:16.229879 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:16.229879 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:16.229879 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:16.229879 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:16.229879 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:16.230633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:16.230633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:16.230633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:16.230633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:16.230633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:16.230633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:16.230633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:16.230633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:16.230633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:16.230633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:16.230633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:16.230633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:16.231290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:16.231290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:16.231290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:16.231290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:16.231290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:16.231290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:16.231290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:16.231290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:16.231290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:16.231290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:16.231290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:16.231290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:16.231290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:16.231909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:16.232618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:16.233495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:16.233495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:16.233495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:16.233495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:16.233495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:16.233495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:18.888693 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:18.889496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:18.889496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:18.890790 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:18.891109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:18.892829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.892829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:18.892829 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.892829 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:18.892829 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:18.892829 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.892829 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:18.892829 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.892829 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:18.893901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.893901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:18.893901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:18.893901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:18.893901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:18.893901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:18.893901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:18.893901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.893901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:18.893901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:18.893901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:18.893901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.894554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:18.894554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:18.894554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:18.894554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:18.894554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:18.894554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:18.894554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:18.894554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:18.894554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.894554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:18.894554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:18.894554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.894554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:18.895238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.895238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:18.895788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:18.896818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.897967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:18.897967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.897967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:18.898666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:18.899011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:18.899134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:18.899318 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:59:18.899669 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:18.899859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:18.900363 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.900363 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:18.900363 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:18.900363 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.900363 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:18.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:18.901333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:18.901607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:18.901607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:18.901607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.901607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:18.901607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:18.901607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.901607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:18.901607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.901607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:18.902305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.902305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:18.902305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:18.902305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:18.902305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:18.902305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:18.902305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:18.902305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:59:18.902305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:18.902305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:18.902305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.902305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:18.902551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:18.902797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.902797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:18.902797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.902797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:18.903703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:18.904000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:18.904354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.904354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:18.904354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:18.904354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.904354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:18.904354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.904354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:18.905500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:18.905734 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:18.905852 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:18.905979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:18.906376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:18.906596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.907097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:18.907097 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.907097 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:18.907097 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:18.907097 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.907097 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:18.907097 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.907097 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:18.907817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:18.908250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:18.908250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:18.908250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.908250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:18.908250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:18.908250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.908250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:18.908250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.908250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:18.908689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.908689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:18.908689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:18.908689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:18.908689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:18.908689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:18.908689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:18.908689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:18.908689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:18.908689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.908689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:18.908977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:18.909354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.909354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:18.909354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.909354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:18.909781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:18.910324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:18.910859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.910859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:18.910859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.910859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:18.915224 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:18.915415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:18.915532 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:18.915687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:18.916216 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:18.916700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:18.917004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.917847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:18.917847 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.917847 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:18.917847 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:18.917847 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.917847 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:18.917847 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.917847 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:18.918792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.918792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:18.918792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:18.918792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:18.918792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:18.918792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:18.918792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:18.918792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.918792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:18.918792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:18.918792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:18.918792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.919245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:18.919245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:18.919245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:18.919245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:18.919245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:18.919245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:18.919245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:18.919245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:18.919245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.919245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:18.919245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:18.919245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.919245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:18.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.919582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:18.920443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:18.920876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:18.921348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:18.921348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:18.921348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:21.196194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:21.196881 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:21.197002 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:21.198261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:21.233914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:21.234427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:21.235338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:21.235338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:21.235338 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:21.235338 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:21.235338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:21.235338 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:21.235338 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:21.235338 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:21.236112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:21.236112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:21.236112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:21.236112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:21.236112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:21.236112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:21.236112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:21.236112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:21.236112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:21.236112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:21.236112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:21.236112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:21.236525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:21.236525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:21.236525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:21.236525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:21.236525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:21.236525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:21.236525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:21.236525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:21.236525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:21.236525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:21.236525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:21.236525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:21.236525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:21.236910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:21.237414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:21.237943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:21.237943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:21.237943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:21.237943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:21.237943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:21.237943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:23.889374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:23.890334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:23.890334 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:23.891154 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:23.891481 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:23.891721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.892682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:23.892682 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.892682 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:23.892682 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:23.892682 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.892682 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:23.892682 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.892682 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:23.894129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.894129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:23.894129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:23.894129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:23.894129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:23.894129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:23.894129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:23.894129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.894129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:23.894129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:23.894129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:23.894129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.895030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:23.895030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:23.895030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:23.895030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:23.895030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:23.895030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:23.895030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:23.895030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:23.895030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.895030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:23.895030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:23.895030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.895030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:23.895719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.895719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:23.895894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:23.897011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.898141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:23.898141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.898141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:23.899011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:23.899325 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:23.899430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:23.899554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:59:23.899999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:23.900400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.901349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:23.901349 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.901349 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:23.901349 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:23.901349 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.901349 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:23.901349 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.901349 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:23.902046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:23.902576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:23.902576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:23.902576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.902576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:23.902576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:23.902576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.902576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:23.902576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.902576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:23.903002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.903002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:23.903002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:23.903002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:23.903002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:23.903002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:23.903002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:23.903002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:59:23.903002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:23.903002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:23.903002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.903002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:23.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:23.903807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.903807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:23.903807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.903807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:23.903982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:23.904490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:23.905009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.905009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:23.905009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:23.905009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.905009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:23.905009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.905009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:23.905862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:23.906118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:23.906284 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:23.906357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:23.906843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:23.907261 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.908033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:23.908033 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.908033 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:23.908033 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:23.908033 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.908033 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:23.908033 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.908033 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:23.908798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:23.909238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:23.909238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:23.909238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.909238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:23.909238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:23.909238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.909238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:23.909238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.909238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:23.909651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.909651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:23.909651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:23.909651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:23.909651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:23.909651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:23.909651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:23.909651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:23.909651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:23.909651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.909651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:23.910049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:23.910451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.910451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:23.910451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.910451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:23.910772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:23.911283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:23.911859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.911859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:23.911859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.911859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:23.915616 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:23.915901 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:23.916035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:23.916223 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:23.916659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:23.917024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:23.917288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.917944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:23.917944 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.917944 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:23.917944 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:23.917944 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.917944 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:23.917944 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.917944 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:23.918747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.918747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:23.918747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:23.918747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:23.918747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:23.918747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:23.918747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:23.918747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.918747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:23.918747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:23.918747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:23.918747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.919131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:23.919131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:23.919131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:23.919131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:23.919131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:23.919131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:23.919131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:23.919131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:23.919131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.919131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:23.919131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:23.919131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.919131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:23.919544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.919544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:23.919712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:23.920297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:23.920851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:23.920851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:23.920851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:26.196878 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:26.197953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:26.197953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:26.198912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:26.233123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:26.233542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:26.234428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:26.234428 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:26.234428 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:26.234428 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:26.234428 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:26.234428 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:26.234428 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:26.234428 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:26.235490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:26.235490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:26.235490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:26.235490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:26.235490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:26.235490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:26.235490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:26.235490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:26.235490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:26.235490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:26.235490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:26.235490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:26.236108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:26.236108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:26.236108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:26.236108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:26.236108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:26.236108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:26.236108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:26.236108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:26.236108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:26.236108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:26.236108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:26.236108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:26.236108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:26.236764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:26.237538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:26.238423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:26.238423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:26.238423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:26.238423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:26.238423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:26.238423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:28.890131 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:28.890685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:28.891385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:28.891944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:28.892299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:28.892560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.893512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:28.893512 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.893512 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:28.893512 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:28.893512 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.893512 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:28.893512 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.893512 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:28.896425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.896425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:28.896425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:28.896425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:28.896425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:28.896425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:28.896425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:28.896425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.896425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:28.896425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:28.896425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:28.896425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.896809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:28.896809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:28.896809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:28.896809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:28.896809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:28.896809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:28.896809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:28.896809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:28.896809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.896809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:28.896809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:28.896809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.896809 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:28.897232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:28.897642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.898158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:28.898158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:28.898158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.898158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:28.898158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.898158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:28.899514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:28.900087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:28.900414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:28.900641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:59:28.901220 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:28.901516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.902666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:28.902666 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.902666 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:28.902666 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:28.902666 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.902666 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:28.902666 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.902666 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:28.903576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:28.903969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:28.903969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:28.903969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.903969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:28.903969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:28.903969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.903969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:28.903969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.903969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:28.904486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.904486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:28.904486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:28.904486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:28.904486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:28.904486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:28.904486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:28.904486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:59:28.904486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:28.904486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:28.904486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.904486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:28.904833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:28.905176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.905176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:28.905176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.905176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:28.905598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:28.906184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:28.906863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.906863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:28.906863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:28.906863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.906863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:28.906863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.906863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:28.906863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:28.906863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:28.906863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:28.907267 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:28.907267 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:28.907364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.907943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:28.907943 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.907943 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:28.907943 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:28.907943 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.907943 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:28.907943 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.907943 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:28.908894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:28.909466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:28.909466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:28.909466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.909466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:28.909466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:28.909466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.909466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:28.909466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.909466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:28.909804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.909804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:28.909804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:28.909804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:28.909804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:28.909804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:28.909804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:28.909804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:28.909804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:28.909804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.909804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:28.910343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:28.910882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.910882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:28.910882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.910882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:28.911057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:28.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:28.912416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.912416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:28.912416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.912416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:28.916167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:28.916512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:28.916622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:28.916795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:28.917120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:28.917373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:28.917515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.918047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:28.918047 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.918047 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:28.918047 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:28.918047 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.918047 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:28.918047 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.918047 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:28.918945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.918945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:28.918945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:28.918945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:28.918945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:28.918945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:28.918945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:28.918945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.918945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:28.918945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:28.918945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:28.918945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.919310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:28.919310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:28.919310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:28.919310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:28.919310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:28.919310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:28.919310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:28.919310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:28.919310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.919310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:28.919310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:28.919310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.919310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:28.919653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.919653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:28.919961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:28.920367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:28.920816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:28.920816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:28.920816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:31.197751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:31.198691 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:31.198691 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:31.199035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:31.231532 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:31.232698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:31.233642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:31.233642 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:31.233642 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:31.233642 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:31.233642 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:31.233642 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:31.233642 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:31.233642 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:31.234425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:31.234425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:31.234425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:31.234425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:31.234425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:31.234425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:31.234425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:31.234425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:31.234425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:31.234425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:31.234425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:31.234425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:31.234815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:31.234815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:31.234815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:31.234815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:31.234815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:31.234815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:31.234815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:31.234815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:31.234815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:31.234815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:31.234815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:31.234815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:31.234815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:31.235197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:31.235197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:31.235295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:31.235753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:31.236306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:31.236306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:31.236306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:33.891659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:33.892864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:33.892864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:33.893245 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:33.893617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:33.893943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.894790 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:33.894790 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.894790 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:33.894790 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:33.894790 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.894790 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:33.894790 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.894790 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:33.895668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.895668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:33.895668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:33.895668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:33.895668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:33.895668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:33.895668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:33.895668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.895668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:33.895668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:33.895668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:33.895668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.896511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:33.896511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:33.896511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:33.896511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:33.896511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:33.896511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:33.896511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:33.896511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:33.896511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.896511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:33.896511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:33.896511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.896511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:33.897237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:33.897864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.900435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:33.900435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:33.900435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.900435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:33.900435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.900435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:33.900435 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:33.900435 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:33.900435 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:33.900435 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:59:33.901280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:33.901280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.901470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:33.901470 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.901470 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:33.901470 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:33.901470 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.901470 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:33.901470 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.901470 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:33.902068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:33.903193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:33.904011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:33.904011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:33.904011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:59:33.904011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:33.904011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:33.904011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.904011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:33.904011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:33.904011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:33.904011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.904011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:33.904011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:33.904011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:33.904386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:33.904386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:33.904386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:33.904386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:33.904386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:33.904386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.904386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:33.904386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:33.904386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.904386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:33.904386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.904386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:33.904712 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:33.905180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:33.905731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.905731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:33.905731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:33.905731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.905731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:33.905731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.905731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:33.906555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:33.906759 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:33.906828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:33.906985 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:33.907289 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:33.907500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.907988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:33.907988 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.907988 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:33.907988 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:33.907988 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.907988 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:33.907988 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.907988 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:33.908520 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:33.908785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:33.908785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:33.908785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.908785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:33.908785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:33.908785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.908785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:33.908785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.908785 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:33.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:33.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:33.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:33.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:33.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:33.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:33.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:33.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:33.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:33.909748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:33.910181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:33.910498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:33.910842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:33.910842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:33.910842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:33.910842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:33.910842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.910842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:33.910842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:33.910842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.910842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:33.910842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.910842 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:33.916551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:33.916838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:33.916914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:33.917047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:33.917722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:33.918181 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:33.918492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.919298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:33.919298 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.919298 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:33.919298 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:33.919298 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.919298 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:33.919298 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.919298 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:33.920015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.920015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:33.920015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:33.920015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:33.920015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:33.920015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:33.920015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:33.920015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.920015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:33.920015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:33.920015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:33.920015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.920391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:33.920391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:33.920391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:33.920391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:33.920391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:33.920391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:33.920391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:33.920391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:33.920391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.920391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:33.920391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:33.920391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.920391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:33.920727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:33.921118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:33.921629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:33.921629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:33.921629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:33.921629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:33.921629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:33.921629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:36.198085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:36.198716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:36.198913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:36.200278 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:36.237524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:36.237720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:36.238330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:36.238330 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:36.238330 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:36.238330 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:36.238330 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:36.238330 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:36.238330 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:36.238330 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:36.239445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:36.239445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:36.239445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:36.239445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:36.239445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:36.239445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:36.239445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:36.239445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:36.239445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:36.239445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:36.239445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:36.239445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:36.240092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:36.240092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:36.240092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:36.240092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:36.240092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:36.240092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:36.240092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:36.240092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:36.240092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:36.240092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:36.240092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:36.240092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:36.240092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:36.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:36.241562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:36.242549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:36.242549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:36.242549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:36.242549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:36.242549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:36.242549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:38.892355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:38.892985 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:38.893134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:38.894658 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:38.895168 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:38.895491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.896522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:38.896522 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.896522 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:38.896522 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:38.896522 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.896522 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:38.896522 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.896522 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:38.898070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.898070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:38.898070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:38.898070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:38.898070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:38.898070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:38.898070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:38.898070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.898070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:38.898070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:38.898070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:38.898070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.899515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:38.899515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:38.899515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:38.899515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:38.899515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:38.899515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:38.899515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:38.899515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:38.899515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.899515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:38.899515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:38.899515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.899515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:38.900845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:38.901938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.902651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:38.902651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:38.902651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.902651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:38.902651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.902651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:38.902651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:38.902651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:38.902651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:38.902651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:38.903081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.903711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:38.904519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.904519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:38.904519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.904519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:38.904519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:38.904519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:38.904519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:38.904519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:38.904519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:38.904519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:59:38.904519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:38.904519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:38.904973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:38.905390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.905832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:38.907101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:38.907101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:38.907101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:38.907101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:38.907424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:38.907669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.908110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:38.908110 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.908110 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:38.908110 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:38.908110 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.908110 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:38.908110 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.908110 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:38.908707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:38.909162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:38.909162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:38.909162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.909162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:38.909162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:38.909162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.909162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:38.909162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.909162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:38.909517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.909517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:38.909517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:38.909517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:38.909517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:38.909517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:38.909517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:38.909517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:38.909517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:38.909517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.909517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:38.910037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:38.910631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:38.910916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:38.911262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:38.911262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:38.911262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:38.911262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:38.911262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.911262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:38.911262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:38.911262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.911262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:38.911262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.911262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:38.916851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:38.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:38.917219 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:38.917453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:38.917954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:38.918333 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:38.918566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.919345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:38.919345 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.919345 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:38.919345 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:38.919345 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.919345 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:38.919345 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.919345 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:38.920616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.920616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:38.920616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:38.920616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:38.920616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:38.920616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:38.920616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:38.920616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.920616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:38.920616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:38.920616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:38.920616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.921465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:38.921465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:38.921465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:38.921465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:38.921465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:38.921465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:38.921465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:38.921465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:38.921465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.921465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:38.921465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:38.921465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.921465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:38.922272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:38.923247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:38.924421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:38.924421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:38.924421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:38.924421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:38.924421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:38.924421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:41.198821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:41.199712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:41.199712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:41.200434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:41.236739 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:41.237317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:41.237880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:41.237880 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:41.237880 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:41.237880 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:41.237880 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:41.237880 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:41.237880 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:41.237880 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:41.238615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:41.238615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:41.238615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:41.238615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:41.238615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:41.238615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:41.238615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:41.238615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:41.238615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:41.238615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:41.238615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:41.238615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:41.239056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:41.239056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:41.239056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:41.239056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:41.239056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:41.239056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:41.239056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:41.239056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:41.239056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:41.239056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:41.239056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:41.239056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:41.239056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:41.239508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:41.239987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:41.240555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:41.240555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:41.240555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:41.240555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:41.240555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:41.240555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:43.892998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:43.893939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:43.894135 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:43.895196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:43.895512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:43.895654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.896570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:43.896570 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.896570 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:43.896570 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:43.896570 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.896570 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:43.896570 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.896570 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:43.898519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.898519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:43.898519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:43.898519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:43.898519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:43.898519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:43.898519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:43.898519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.898519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:43.898519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:43.898519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:43.898519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.899329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:43.899329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:43.899329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:43.899329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:43.899329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:43.899329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:43.899329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:43.899329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:43.899329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.899329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:43.899329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:43.899329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.899329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:43.899833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.899833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:43.900678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:43.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.902174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:43.902174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.902174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:43.902174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:43.902174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:43.902174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:43.902174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:59:43.902451 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:43.902728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.903560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:43.903560 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.903560 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:43.903560 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:43.903560 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.903560 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:43.903560 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.903560 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:43.904529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:43.905007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:43.905007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:43.905007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.905007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:43.905007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:43.905007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.905007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:43.905007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.905007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:43.905666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.905666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:43.905666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:43.905666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:43.905666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:43.905666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:43.905666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:43.905666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:59:43.905666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:43.905666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:43.905666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.905666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:43.906079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:43.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:43.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.906505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:43.907041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:43.907562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:43.908829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.908829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:43.908829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:43.908829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.908829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:43.908829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.908829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:43.908829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:43.908829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:43.908829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:43.909265 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:43.909265 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:43.909561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.910472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:43.910472 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.910472 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:43.910472 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:43.910472 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.910472 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:43.910472 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.910472 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:43.911183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:43.911651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:43.911651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:43.911651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.911651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:43.911651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:43.911651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.911651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:43.911651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.911651 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:43.912118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.912118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:43.912118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:43.912118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:43.912118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:43.912118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:43.912118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:43.912118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:43.912118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:43.912118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.912118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:43.912496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:43.912889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.912889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:43.912889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.912889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:43.913170 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:43.913823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:43.914404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.914404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:43.914404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.914404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:43.917263 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:43.917704 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:43.917842 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:43.918074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:43.918450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:43.918670 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:43.918848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.919484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:43.919484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.919484 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:43.919484 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:43.919484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.919484 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:43.919484 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.919484 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:43.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:43.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:43.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:43.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:43.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:43.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:43.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:43.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:43.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:43.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.920967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:43.920967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:43.920967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:43.920967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:43.920967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:43.920967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:43.920967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:43.920967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:43.920967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.920967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:43.920967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:43.920967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.920967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:43.921343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.921343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:43.921882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:43.922402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:43.922967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:43.922967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:43.922967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:46.199588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:46.202523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:46.202523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:46.202523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:46.236057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:46.236446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:46.236664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:46.236664 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:46.236664 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:46.236664 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:46.236664 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:46.236664 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:46.236664 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:46.236664 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:46.237335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:46.237335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:46.237335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:46.237335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:46.237335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:46.237335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:46.237335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:46.237335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:46.237335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:46.237335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:46.237335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:46.237335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:46.237980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:46.237980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:46.237980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:46.237980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:46.237980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:46.237980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:46.237980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:46.237980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:46.237980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:46.237980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:46.237980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:46.237980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:46.237980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:46.238654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:46.239442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:46.240346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:46.240346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:46.240346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:46.240346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:46.240346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:46.240346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:48.893997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:48.894844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:48.894844 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:48.895653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:48.895924 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:48.896152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.896946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:48.896946 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.896946 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:48.896946 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:48.896946 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.896946 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:48.896946 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.896946 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:48.898923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.898923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:48.898923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:48.898923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:48.898923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:48.898923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:48.898923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:48.898923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.898923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:48.898923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:48.898923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:48.898923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.899562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:48.899562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:48.899562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:48.899562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:48.899562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:48.899562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:48.899562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:48.899562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:48.899562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.899562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:48.899562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:48.899562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.899562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:48.900117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.900117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:48.900610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:48.901371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.902230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:48.902230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.902230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:48.902230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:48.902230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:48.902230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:48.902230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:59:48.902603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:48.902603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.903072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:48.903072 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.903072 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:48.903072 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:48.903072 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.903072 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:48.903072 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.903072 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:48.903837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:48.904335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:48.904335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:48.904335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.904335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:48.904335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:48.904335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.904335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:48.904335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.904335 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:48.904723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.904723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:48.904723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:48.904723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:48.904723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:48.904723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:48.904723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:48.904723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:59:48.904723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:48.904723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:48.904723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.904723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:48.905089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:48.905512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.905512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:48.905512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.905512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:48.905816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:48.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:48.906795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.906795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:48.906795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:48.906795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.906795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:48.906795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.906795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:48.908317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:48.908531 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:48.908648 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:48.908789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:48.909013 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:48.909315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.909743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:48.909743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.909743 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:48.909743 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:48.909743 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.909743 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:48.909743 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.909743 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:48.910495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:48.910932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:48.910932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:48.910932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.910932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:48.910932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:48.910932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.910932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:48.910932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.910932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:48.911410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.911410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:48.911410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:48.911410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:48.911410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:48.911410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:48.911410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:48.911410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:48.911410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:48.911410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.911410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:48.911754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:48.912132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.912132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:48.912132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.912132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:48.912515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:48.912989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:48.913526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.913526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:48.913526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.913526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:48.917624 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:48.917902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:48.918013 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:48.918166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:48.918421 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:48.918641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:48.918777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.919253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:48.919253 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.919253 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:48.919253 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:48.919253 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.919253 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:48.919253 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.919253 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:48.920067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.920067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:48.920067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:48.920067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:48.920067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:48.920067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:48.920067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:48.920067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.920067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:48.920067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:48.920067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:48.920067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.920451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:48.920451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:48.920451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:48.920451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:48.920451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:48.920451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:48.920451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:48.920451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:48.920451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.920451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:48.920451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:48.920451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.920451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:48.920788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.920788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:48.921071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:48.921555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:48.922055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:48.922055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:48.922055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:51.200158 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:51.201018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:51.201018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:51.202088 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:51.238408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:51.238408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:51.238798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:51.238798 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:51.238798 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:51.238798 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:51.238798 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:51.238798 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:51.238798 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:51.238798 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:51.239466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:51.239466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:51.239466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:51.239466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:51.239466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:51.239466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:51.239466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:51.239466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:51.239466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:51.239466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:51.239466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:51.239466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:51.240134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:51.240134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:51.240134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:51.240134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:51.240134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:51.240134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:51.240134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:51.240134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:51.240134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:51.240134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:51.240134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:51.240134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:51.240134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:51.240767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:51.241504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:51.242320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:51.242320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:51.242320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:51.242320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:51.242320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:51.242320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:53.894743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:53.895831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:53.895831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:53.896483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:53.896834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:53.897144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.898044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:53.898044 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.898044 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:53.898044 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:53.898044 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.898044 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:53.898044 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.898044 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:53.899734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.899734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:53.899734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:53.899734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:53.899734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:53.899734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:53.899734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:53.899734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.899734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:53.899734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:53.899734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:53.899734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.900734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:53.900734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:53.900734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:53.900734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:53.900734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:53.900734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:53.900734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:53.900734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:53.900734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.900734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:53.900734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:53.900734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.900734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:53.901884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.901884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:53.902121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:53.903909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.904626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:53.904626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.904626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:53.904626 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:53.904626 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:53.904626 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:53.904626 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:59:53.904626 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:53.904906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.904906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:53.904906 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.904906 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:53.904906 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:53.904906 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.904906 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:53.904906 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.904906 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:53.905252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:53.905928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:53.905928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:53.905928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.905928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:53.905928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:53.905928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.905928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:53.905928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.905928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:53.906299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.906299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:53.906299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:53.906299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:53.906299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:53.906299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:53.906299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:53.906299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:59:53.906299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:53.906299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:53.906299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.906299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:53.906664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:53.907185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:53.907488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:53.907847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:53.907847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:53.907847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:53.907847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:53.907847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:53.907847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.907847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:53.907847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:53.907847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.907847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:53.907847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.907847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:53.908569 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:53.908722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:53.908831 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:53.908963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:53.909170 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:53.909391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:53.909758 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.909758 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:53.909758 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:53.909758 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.909758 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:53.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:53.910363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:53.910818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:53.910818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:53.910818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.910818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:53.910818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:53.910818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.910818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:53.910818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.910818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:53.911272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.911272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:53.911272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:53.911272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:53.911272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:53.911272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:53.911272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:53.911272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:53.911272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:53.911272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.911272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:53.911639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:53.912070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.912070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:53.912070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.912070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:53.912342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:53.912729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:53.913089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.913089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:53.913089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.913089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:53.918273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:53.918552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:53.918652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:53.918797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:53.919022 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:53.919217 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:53.919359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.919731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:53.919731 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.919731 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:53.919731 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:53.919731 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.919731 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:53.919731 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.919731 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:53.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:53.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:53.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:53.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:53.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:53.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:53.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:53.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:53.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:53.920577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.920955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:53.920955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:53.920955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:53.920955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:53.920955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:53.920955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:53.920955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:53.920955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:53.920955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.920955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:53.920955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:53.920955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.920955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:53.921395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.921395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:53.921564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:53.921927 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:53.922475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:53.922475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:53.922475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:56.200805 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:56.201897 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:56.201897 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:56.203104 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:56.238773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:56.238953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:56.240340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:56.240340 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:56.240340 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:56.240340 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:56.240340 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:56.240340 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:56.240340 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:56.240340 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:56.241026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:56.241026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:56.241026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:56.241026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:56.241026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:56.241026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:56.241026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:56.241026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:56.241026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:56.241026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:56.241026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:56.241026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:56.241557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:56.241557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:56.241557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:56.241557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:56.241557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:56.241557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:56.241557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:56.241557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:56.241557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:56.241557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:56.241557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:56.241557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:56.241557 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:56.242159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:56.242739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:56.244511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:56.244511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:56.244511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:56.244511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:56.244511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:56.244511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:58.895327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:58.896140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:58.896140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:58.897963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:58.898434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:58.898696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.899682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:58.899682 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.899682 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:58.899682 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:58.899682 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.899682 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:58.899682 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.899682 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:58.901336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.901336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:58.901336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:58.901336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:58.901336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:58.901336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:58.901336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:58.901336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.901336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:58.901336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:58.901336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:58.901336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.901946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:58.901946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:58.901946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:58.901946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:58.901946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:58.901946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:58.901946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:58.901946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:58.901946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.901946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:58.901946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:58.901946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.901946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:58.902529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.902529 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:58.903294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:58.903967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.904804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:58.904804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.904804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:58.904804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:58.904804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:58.904804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:58.904804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 15:59:58.905142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:58.905313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.906053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:58.906053 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.906053 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:58.906053 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:58.906053 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.906053 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:58.906053 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.906053 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:58.906823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:58.907233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:58.907233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:58.907233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.907233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:58.907233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:58.907233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.907233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:58.907233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.907233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:58.907614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.907614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:58.907614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:58.907614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:58.907614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:58.907614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:58.907614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:58.907614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 15:59:58.907614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:58.907614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:58.907614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.907614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:58.907935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:58.908293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.908293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:58.908293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.908293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:58.908581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:58.908989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:58.909457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.909457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:58.909457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:58.909457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.909457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:58.909457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.909457 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:58.909457 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:58.909689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:58.909689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:58.909804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:58.910222 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:58.910610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.911312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:58.911312 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.911312 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:58.911312 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:58.911312 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.911312 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:58.911312 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.911312 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 15:59:58.912020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 15:59:58.912442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:58.912442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 15:59:58.912442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.912442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 15:59:58.912442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 15:59:58.912442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.912442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 15:59:58.912442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.912442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 15:59:58.912892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.912892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:58.912892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:58.912892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:58.912892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:58.912892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:58.912892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:58.912892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:58.912892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:58.912892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.912892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:58.913386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:58.913726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.913726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:58.913726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.913726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:58.913898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:58.914394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:58.914876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.914876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:58.914876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.914876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 15:59:58.918603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 15:59:58.918818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 15:59:58.918925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 15:59:58.919069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 15:59:58.919510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:58.919867 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 15:59:58.920095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.920887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 15:59:58.920887 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.920887 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 15:59:58.920887 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 15:59:58.920887 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.920887 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 15:59:58.920887 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.920887 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 15:59:58.921676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.921676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 15:59:58.921676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:58.921676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 15:59:58.921676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 15:59:58.921676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:58.921676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:58.921676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.921676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:58.921676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:58.921676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 15:59:58.921676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.922038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 15:59:58.922038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:58.922038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 15:59:58.922038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:58.922038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 15:59:58.922038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 15:59:58.922038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:58.922038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 15:59:58.922038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.922038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 15:59:58.922038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 15:59:58.922038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.922038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 15:59:58.922411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.922411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 15:59:58.922613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 15:59:58.923054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 15:59:58.923607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 15:59:58.923607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 15:59:58.923607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:01.201403 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:01.202162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:01.202162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:01.203377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:01.240238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:01.240673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:01.241603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:01.241603 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:01.241603 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:01.241603 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:01.241603 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:01.241603 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:01.241603 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:01.241603 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:01.242741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:01.242741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:01.242741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:01.242741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:01.242741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:01.242741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:01.242741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:01.242741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:01.242741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:01.242741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:01.242741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:01.242741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:01.243786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:01.243786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:01.243786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:01.243786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:01.243786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:01.243786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:01.243786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:01.243786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:01.243786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:01.243786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:01.243786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:01.243786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:01.243786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:01.244503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:01.245254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:01.246072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:01.246072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:01.246072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:01.246072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:01.246072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:01.246072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:03.896019 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:03.896963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:03.896963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:03.897429 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:03.897748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:03.897981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.898907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:03.898907 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.898907 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:03.898907 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:03.898907 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.898907 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:03.898907 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.898907 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:03.899911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.899911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:03.899911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:03.899911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:03.899911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:03.899911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:03.899911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:03.899911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.899911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:03.899911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:03.899911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:03.899911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:03.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:03.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:03.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:03.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:03.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:03.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:03.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:03.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:03.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:03.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.900363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:03.900843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.900843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:03.901305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:03.902755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.903915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:03.903915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.903915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:03.904367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:03.904747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:03.904853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:03.904982 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:00:03.905649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:03.906002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.906825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:03.906825 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.906825 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:03.906825 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:03.906825 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.906825 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:03.906825 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.906825 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:03.907878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:03.908430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:03.908430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:03.908430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.908430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:03.908430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:03.908430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.908430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:03.908430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.908430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:03.908699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.908699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:03.908699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:03.908699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:03.908699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:03.908699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:03.908699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:03.908699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:00:03.908699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:03.908699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:03.908699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.908699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:03.909079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:03.909458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.909458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:03.909458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.909458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:03.910044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:03.911029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:03.912125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.912125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:03.912125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:03.912125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.912125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:03.912125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.912125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:03.912125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:03.912125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:03.912125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:03.912737 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:03.912737 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:03.912737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.912737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:03.912737 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.912737 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:03.912737 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:03.912737 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.912737 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:03.912737 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.912737 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:03.913403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:03.914128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:03.914770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:03.914770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:03.914770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:03.914770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:03.914770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.914770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:03.914770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:03.914770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:03.914770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.914770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:03.914770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:03.914770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:03.914770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:03.915156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:03.915156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:03.915156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:03.915156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:03.915156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.915156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:03.915156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:03.915156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.915156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:03.915156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.915156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:03.915156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:03.915538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.916026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:03.918896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:03.919104 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:03.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:03.919407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:03.919786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:03.920015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:03.920165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.920852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:03.920852 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.920852 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:03.920852 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:03.920852 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.920852 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:03.920852 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.920852 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:03.921737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.921737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:03.921737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:03.921737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:03.921737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:03.921737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:03.921737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:03.921737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.921737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:03.921737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:03.921737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:03.921737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:03.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:03.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:03.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:03.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:03.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:03.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:03.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:03.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:03.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:03.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:03.922571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.922571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:03.922897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:03.923899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:03.924890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:03.924890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:03.924890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:06.202262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:06.202262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:06.202857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:06.203153 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:06.244654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:06.245052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:06.245986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:06.245986 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:06.245986 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:06.245986 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:06.245986 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:06.245986 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:06.245986 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:06.245986 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:06.247104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:06.247104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:06.247104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:06.247104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:06.247104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:06.247104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:06.247104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:06.247104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:06.247104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:06.247104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:06.247104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:06.247104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:06.247541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:06.247541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:06.247541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:06.247541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:06.247541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:06.247541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:06.247541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:06.247541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:06.247541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:06.247541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:06.247541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:06.247541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:06.247541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:06.247924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:06.247924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:06.249264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:06.250000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:06.250563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:06.250563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:06.250563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:08.896634 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:08.897738 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:08.897867 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:08.898808 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:08.899012 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:08.899172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.899861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:08.899861 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.899861 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:08.899861 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:08.899861 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.899861 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:08.899861 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.899861 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:08.900925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.900925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:08.900925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:08.900925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:08.900925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:08.900925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:08.900925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:08.900925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.900925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:08.900925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:08.900925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:08.900925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.901358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:08.901358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:08.901358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:08.901358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:08.901358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:08.901358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:08.901358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:08.901358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:08.901358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.901358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:08.901358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:08.901358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.901358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:08.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.901643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:08.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:08.902424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.902854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:08.902854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.902854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:08.905099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:08.905418 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:08.905585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:08.905751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:00:08.906045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:08.906321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.906798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:08.906798 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.906798 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:08.906798 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:08.906798 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.906798 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:08.906798 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.906798 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:08.907581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:08.907951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:08.907951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:08.907951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.907951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:08.907951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:08.907951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.907951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:08.907951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.907951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:08.908901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.908901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:08.908901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:08.908901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:08.908901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:08.908901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:08.908901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:08.908901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:00:08.908901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:08.908901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:08.908901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.908901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:08.909187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:08.909478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:08.909753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:08.910120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:08.910120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:08.910120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:08.910120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:08.910120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:08.910120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.910120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:08.910120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:08.910120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.910120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:08.910120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.910120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:08.910352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:08.910550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:08.910645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:08.910758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:08.911741 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:08.912104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.912752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:08.912752 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.912752 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:08.912752 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:08.912752 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.912752 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:08.912752 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.912752 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:08.913515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:08.913814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:08.913814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:08.913814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.913814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:08.913814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:08.913814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.913814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:08.913814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.913814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:08.914397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.914397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:08.914397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:08.914397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:08.914397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:08.914397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:08.914397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:08.914397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:08.914397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:08.914397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.914397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:08.914659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:08.914931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.914931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:08.914931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.914931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:08.915476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:08.915840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:08.916225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.916225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:08.916225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.916225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:08.919293 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:08.919551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:08.919663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:08.919840 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:08.920350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:08.920641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:08.920849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.921592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:08.921592 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.921592 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:08.921592 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:08.921592 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.921592 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:08.921592 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.921592 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:08.922439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.922439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:08.922439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:08.922439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:08.922439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:08.922439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:08.922439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:08.922439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.922439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:08.922439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:08.922439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:08.922439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.922679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:08.922679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:08.922679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:08.922679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:08.922679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:08.922679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:08.922679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:08.922679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:08.922679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.922679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:08.922679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:08.922679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.922679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:08.922905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.922905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:08.923435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:08.923741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:08.924108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:08.924108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:08.924108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:11.202416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:11.203348 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:11.203348 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:11.204427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:11.211674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 16:00:11.239961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:11.240272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:11.240952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:11.240952 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:11.240952 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:11.240952 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:11.240952 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:11.240952 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:11.240952 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:11.240952 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:11.241708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:11.241708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:11.241708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:11.241708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:11.241708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:11.241708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:11.241708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:11.241708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:11.241708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:11.241708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:11.241708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:11.241708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:11.242546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:11.242546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:11.242546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:11.242546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:11.242546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:11.242546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:11.242546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:11.242546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:11.242546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:11.242546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:11.242546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:11.242546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:11.242546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:11.243334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:11.244318 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:11.245596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:11.245596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:11.245596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:11.245596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:11.245596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:11.245596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:13.897852 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:13.898513 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:13.898513 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:13.899203 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:13.899523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:13.899767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.900658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:13.900658 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.900658 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:13.900658 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:13.900658 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.900658 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:13.900658 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.900658 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:13.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:13.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:13.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:13.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:13.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:13.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:13.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:13.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:13.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:13.902537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.903445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:13.903445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:13.903445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:13.903445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:13.903445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:13.903445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:13.903445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:13.903445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:13.903445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.903445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:13.903445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:13.903445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.903445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:13.903841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.903841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:13.904594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:13.905185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.905746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:13.905746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.905746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:13.905746 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:13.905874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:13.906131 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:13.906488 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:00:13.906599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:13.906741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.907274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:13.907274 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.907274 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:13.907274 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:13.907274 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.907274 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:13.907274 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.907274 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:13.908083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:13.909006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:13.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:13.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:13.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:00:13.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:13.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:13.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:13.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:13.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:13.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:13.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:13.909758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:13.910108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:13.910108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:13.910108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:13.910108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:13.910108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:13.910108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.910108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:13.910108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:13.910108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.910108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:13.910108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.910108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:13.910423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:13.910827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:13.911327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.911327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:13.911327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:13.911327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.911327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:13.911327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.911327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:13.911327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:13.911327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:13.911327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:13.911623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:13.911662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:13.911865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.912329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:13.912329 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.912329 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:13.912329 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:13.912329 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.912329 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:13.912329 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.912329 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:13.913012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:13.913451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:13.913451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:13.913451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.913451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:13.913451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:13.913451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.913451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:13.913451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.913451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:13.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:13.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:13.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:13.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:13.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:13.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:13.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:13.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:13.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.913914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:13.914274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:13.914653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.914653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:13.914653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.914653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:13.915021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:13.915577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:13.916165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.916165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:13.916165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.916165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:13.919696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:13.919956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:13.920067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:13.920218 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:13.920622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:13.920976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:13.921198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:13.921912 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.921912 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:13.921912 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:13.921912 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.921912 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:13.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:13.922803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.922803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:13.922803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:13.922803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:13.922803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:13.922803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:13.922803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:13.922803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.922803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:13.922803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:13.922803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:13.922803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.923174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:13.923174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:13.923174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:13.923174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:13.923174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:13.923174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:13.923174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:13.923174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:13.923174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.923174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:13.923174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:13.923174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.923174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:13.923566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.923566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:13.924010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:13.924503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:13.925000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:13.925000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:13.925000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:16.203005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:16.203596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:16.203757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:16.205059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:16.246459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:16.246724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:16.247425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:16.247425 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:16.247425 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:16.247425 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:16.247425 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:16.247425 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:16.247425 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:16.247425 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:16.248159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:16.248159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:16.248159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:16.248159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:16.248159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:16.248159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:16.248159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:16.248159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:16.248159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:16.248159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:16.248159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:16.248159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:16.248741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:16.248741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:16.248741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:16.248741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:16.248741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:16.248741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:16.248741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:16.248741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:16.248741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:16.248741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:16.248741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:16.248741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:16.248741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:16.249265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:16.249879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:16.250707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:16.250707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:16.250707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:16.250707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:16.250707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:16.250707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:18.898506 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:18.899310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:18.899310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:18.900300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:18.900617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:18.900880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.901714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:18.901714 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.901714 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:18.901714 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:18.901714 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.901714 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:18.901714 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.901714 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:18.903428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.903428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:18.903428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:18.903428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:18.903428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:18.903428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:18.903428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:18.903428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.903428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:18.903428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:18.903428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:18.903428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.903849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:18.903849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:18.903849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:18.903849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:18.903849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:18.903849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:18.903849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:18.903849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:18.903849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.903849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:18.903849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:18.903849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.903849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:18.904200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.904200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:18.905128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:18.905599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.906128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:18.906128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.906128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:18.906128 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:18.906267 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:18.906307 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:18.906451 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:00:18.906860 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:18.907134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.907846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:18.907846 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.907846 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:18.907846 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:18.907846 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.907846 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:18.907846 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.907846 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:18.908608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:18.909003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:18.909003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:18.909003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.909003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:18.909003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:18.909003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.909003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:18.909003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.909003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:18.909575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.909575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:18.909575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:18.909575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:18.909575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:18.909575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:18.909575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:18.909575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:00:18.909575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:18.909575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:18.909575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.909575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:18.909928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:18.910322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.910322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:18.910322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.910322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:18.911026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:18.911795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:18.912313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.912313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:18.912313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:18.912313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.912313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:18.912313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.912313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:18.912313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:18.912313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:18.912313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:18.912764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:18.912764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:18.912764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.913279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:18.913279 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.913279 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:18.913279 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:18.913279 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.913279 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:18.913279 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.913279 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:18.914033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:18.914532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:18.914532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:18.914532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.914532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:18.914532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:18.914532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.914532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:18.914532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.914532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:18.915026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.915026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:18.915026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:18.915026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:18.915026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:18.915026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:18.915026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:18.915026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:18.915026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:18.915026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.915026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:18.915412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:18.915847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.915847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:18.915847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.915847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:18.916282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:18.917513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:18.918034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.918034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:18.918034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.918034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:18.920111 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:18.920397 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:18.920507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:18.920670 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:18.921083 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:18.921471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:18.921670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.922316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:18.922316 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.922316 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:18.922316 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:18.922316 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.922316 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:18.922316 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.922316 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:18.923156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.923156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:18.923156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:18.923156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:18.923156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:18.923156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:18.923156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:18.923156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.923156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:18.923156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:18.923156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:18.923156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.923577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:18.923577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:18.923577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:18.923577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:18.923577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:18.923577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:18.923577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:18.923577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:18.923577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.923577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:18.923577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:18.923577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.923577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:18.923976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.923976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:18.924169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:18.924710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:18.925359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:18.925359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:18.925359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:21.203921 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:21.204603 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:21.204800 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:21.206105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:21.242711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:21.243125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:21.243935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:21.243935 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:21.243935 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:21.243935 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:21.243935 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:21.243935 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:21.243935 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:21.243935 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:21.244679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:21.244679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:21.244679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:21.244679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:21.244679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:21.244679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:21.244679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:21.244679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:21.244679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:21.244679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:21.244679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:21.244679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:21.245532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:21.245532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:21.245532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:21.245532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:21.245532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:21.245532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:21.245532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:21.245532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:21.245532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:21.245532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:21.245532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:21.245532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:21.245532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:21.246254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:21.247095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:21.248034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:21.248034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:21.248034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:21.248034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:21.248034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:21.248034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:23.899392 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:23.900484 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:23.900484 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:23.901452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:23.901808 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:23.902003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.902795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:23.902795 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.902795 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:23.902795 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:23.902795 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.902795 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:23.902795 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.902795 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:23.904759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.904759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:23.904759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:23.904759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:23.904759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:23.904759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:23.904759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:23.904759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.904759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:23.904759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:23.904759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:23.904759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.905760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:23.905760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:23.905760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:23.905760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:23.905760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:23.905760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:23.905760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:23.905760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:23.905760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.905760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:23.905760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:23.905760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.905760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:23.906401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.906401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:23.907408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:23.908374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.909369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:23.909369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.909369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:23.909369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:23.909369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:23.909369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:23.909369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:00:23.909369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:23.909838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.909838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:23.909838 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.909838 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:23.909838 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:23.909838 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.909838 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:23.909838 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.909838 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:23.910816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:23.911699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:23.912171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:23.912171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:23.912171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:00:23.912171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:23.912171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:23.912171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.912171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:23.912171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:23.912171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:23.912171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.912171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:23.912171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:23.912171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:23.912612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:23.912612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:23.912612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:23.912612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:23.912612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:23.912612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.912612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:23.912612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:23.912612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.912612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:23.912612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.912612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:23.912922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:23.913508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:23.914006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.914006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:23.914006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:23.914006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.914006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:23.914006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.914006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:23.914006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:23.914006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:23.914006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:23.914503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:23.914503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:23.914759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.915597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:23.915597 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.915597 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:23.915597 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:23.915597 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.915597 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:23.915597 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.915597 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:23.916413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:23.916888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:23.916888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:23.916888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.916888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:23.916888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:23.916888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.916888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:23.916888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.916888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:23.917279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.917279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:23.917279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:23.917279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:23.917279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:23.917279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:23.917279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:23.917279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:23.917279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:23.917279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.917279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:23.917672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:23.918103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.918103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:23.918103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.918103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:23.918424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:23.919427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:23.919969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.919969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:23.919969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.919969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:23.920526 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:23.920859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:23.920915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:23.921108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:23.921433 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:23.921680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:23.921847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.922380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:23.922380 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.922380 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:23.922380 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:23.922380 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.922380 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:23.922380 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.922380 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:23.923277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.923277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:23.923277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:23.923277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:23.923277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:23.923277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:23.923277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:23.923277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.923277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:23.923277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:23.923277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:23.923277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.923781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:23.923781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:23.923781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:23.923781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:23.923781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:23.923781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:23.923781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:23.923781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:23.923781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.923781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:23.923781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:23.923781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.923781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:23.924147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.924147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:23.924320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:23.924784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:23.925356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:23.925356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:23.925356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:26.204741 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:26.205638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:26.205638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:26.206790 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:26.235895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:26.236350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:26.237162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:26.237162 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:26.237162 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:26.237162 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:26.237162 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:26.237162 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:26.237162 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:26.237162 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:26.238239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:26.238239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:26.238239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:26.238239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:26.238239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:26.238239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:26.238239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:26.238239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:26.238239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:26.238239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:26.238239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:26.238239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:26.238942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:26.238942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:26.238942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:26.238942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:26.238942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:26.238942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:26.238942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:26.238942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:26.238942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:26.238942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:26.238942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:26.238942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:26.238942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:26.239660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:26.240383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:26.241197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:26.241197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:26.241197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:26.241197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:26.241197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:26.241197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:28.901379 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:28.902194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:28.902194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:28.902628 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:28.902745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:28.902847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.903358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:28.903358 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.903358 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:28.903358 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:28.903358 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.903358 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:28.903358 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.903358 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:28.904319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.904319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:28.904319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:28.904319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:28.904319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:28.904319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:28.904319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:28.904319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.904319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:28.904319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:28.904319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:28.904319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.904773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:28.904773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:28.904773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:28.904773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:28.904773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:28.904773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:28.904773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:28.904773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:28.904773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.904773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:28.904773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:28.904773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.904773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:28.905193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.905193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:28.905470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:28.906063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.906701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:28.906701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.906701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:28.908009 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:28.908305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:28.908426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:28.908548 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:00:28.908772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:28.908901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.909276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:28.909276 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.909276 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:28.909276 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:28.909276 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.909276 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:28.909276 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.909276 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:28.910107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:28.910733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:28.910733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:28.910733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.910733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:28.910733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:28.910733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.910733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:28.910733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.910733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:28.911041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.911041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:28.911041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:28.911041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:28.911041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:28.911041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:28.911041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:28.911041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:00:28.911041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:28.911041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:28.911041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.911041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:28.911546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:28.911959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.911959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:28.911959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.911959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:28.912092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:28.912628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:28.913156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.913156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:28.913156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:28.913156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.913156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:28.913156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.913156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:28.913728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:28.913988 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:28.914103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:28.914272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:28.914487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:28.914637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.914998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:28.914998 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.914998 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:28.914998 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:28.914998 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.914998 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:28.914998 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.914998 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:28.915768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:28.916243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:28.916243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:28.916243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.916243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:28.916243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:28.916243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.916243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:28.916243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.916243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:28.916696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.916696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:28.916696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:28.916696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:28.916696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:28.916696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:28.916696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:28.916696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:28.916696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:28.916696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.916696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:28.917093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:28.917637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.917637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:28.917637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.917637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:28.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:28.918521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:28.919076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.919076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:28.919076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.919076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:28.920959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:28.921218 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:28.921339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:28.921523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:28.921747 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:28.921892 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:28.921990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.922339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:28.922339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.922339 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:28.922339 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:28.922339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.922339 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:28.922339 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.922339 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:28.923135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.923135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:28.923135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:28.923135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:28.923135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:28.923135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:28.923135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:28.923135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.923135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:28.923135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:28.923135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:28.923135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.923548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:28.923548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:28.923548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:28.923548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:28.923548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:28.923548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:28.923548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:28.923548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:28.923548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.923548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:28.923548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:28.923548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.923548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:28.923890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.923890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:28.924129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:28.924633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:28.925195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:28.925195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:28.925195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:31.205439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:31.206185 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:31.206185 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:31.207339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:31.248993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:31.250159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:31.251004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:31.251004 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:31.251004 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:31.251004 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:31.251004 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:31.251004 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:31.251004 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:31.251004 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:31.252255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:31.252255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:31.252255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:31.252255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:31.252255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:31.252255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:31.252255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:31.252255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:31.252255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:31.252255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:31.252255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:31.252255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:31.252869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:31.252869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:31.252869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:31.252869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:31.252869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:31.252869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:31.252869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:31.252869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:31.252869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:31.252869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:31.252869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:31.252869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:31.252869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:31.253534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:31.254312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:31.255479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:31.255479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:31.255479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:31.255479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:31.255479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:31.255479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:33.902177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:33.902792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:33.902863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:33.903976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:33.904340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:33.904578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.905462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:33.905462 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.905462 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:33.905462 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:33.905462 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.905462 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:33.905462 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.905462 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:33.906936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.906936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:33.906936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:33.906936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:33.906936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:33.906936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:33.906936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:33.906936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.906936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:33.906936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:33.906936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:33.906936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.907646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:33.907646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:33.907646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:33.907646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:33.907646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:33.907646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:33.907646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:33.907646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:33.907646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.907646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:33.907646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:33.907646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.907646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:33.908459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.908459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:33.908693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:33.909537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.910099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:33.910099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.910099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:33.910099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:33.910099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:33.910099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:33.910099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:00:33.910386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:33.910698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.911592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:33.911592 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.911592 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:33.911592 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:33.911592 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.911592 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:33.911592 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.911592 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:33.912377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:33.912755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:33.912755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:33.912755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.912755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:33.912755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:33.912755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.912755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:33.912755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.912755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:33.913267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.913267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:33.913267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:33.913267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:33.913267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:33.913267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:33.913267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:33.913267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:00:33.913267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:33.913267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:33.913267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.913267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:33.913625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:33.913939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.913939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:33.913939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.913939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:33.914485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:33.914873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:33.915451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.915451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:33.915451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:33.915451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.915451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:33.915451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.915451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:33.915640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:33.916092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:33.916150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:33.916371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:33.916688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:33.916935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.918290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:33.918290 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.918290 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:33.918290 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:33.918290 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.918290 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:33.918290 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.918290 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:33.919185 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:33.919643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:33.919643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:33.919643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.919643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:33.919643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:33.919643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.919643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:33.919643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.919643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:33.920623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.920623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:33.920623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:33.920623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:33.920623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:33.920623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:33.920623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:33.920623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:33.920623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:33.920623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.920623 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:33.920980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:33.921481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.921481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:33.921481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.921481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:33.921649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:33.922161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:33.922915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.922915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:33.922915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.922915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:33.922915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:33.922915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:33.923201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:33.923472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:33.923804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:33.924047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:33.924196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.924760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:33.924760 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.924760 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:33.924760 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:33.924760 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.924760 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:33.924760 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.924760 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:33.925616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.925616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:33.925616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:33.925616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:33.925616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:33.925616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:33.925616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:33.925616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.925616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:33.925616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:33.925616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:33.925616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.925937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:33.925937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:33.925937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:33.925937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:33.925937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:33.925937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:33.925937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:33.925937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:33.925937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.925937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:33.925937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:33.925937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.925937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:33.926378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.926378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:33.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:33.927160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:33.927884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:33.927884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:33.927884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:36.206046 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:36.206890 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:36.206890 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:36.207585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:36.239792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:36.240247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:36.241141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:36.241141 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:36.241141 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:36.241141 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:36.241141 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:36.241141 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:36.241141 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:36.241141 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:36.242150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:36.242150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:36.242150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:36.242150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:36.242150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:36.242150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:36.242150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:36.242150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:36.242150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:36.242150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:36.242150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:36.242150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:36.242766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:36.242766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:36.242766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:36.242766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:36.242766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:36.242766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:36.242766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:36.242766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:36.242766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:36.242766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:36.242766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:36.242766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:36.242766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:36.243431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:36.244321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:36.245348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:36.245348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:36.245348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:36.245348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:36.245348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:36.245348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:38.903020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:38.904060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:38.904060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:38.905201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:38.905633 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:38.906525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.906976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:38.906976 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.906976 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:38.906976 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:38.906976 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.906976 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:38.906976 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.906976 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:38.908510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.908510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:38.908510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:38.908510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:38.908510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:38.908510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:38.908510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:38.908510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.908510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:38.908510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:38.908510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:38.908510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.909146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:38.909146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:38.909146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:38.909146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:38.909146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:38.909146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:38.909146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:38.909146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:38.909146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.909146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:38.909146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:38.909146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.909146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:38.909754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.909754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:38.910611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:38.911418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.912288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:38.912288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.912288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:38.912288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:38.912288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:38.912288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:38.912288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:00:38.912288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:38.912788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.913448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:38.913448 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.913448 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:38.913448 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:38.913448 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.913448 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:38.913448 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.913448 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:38.914183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:38.914721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:38.914721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:38.914721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.914721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:38.914721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:38.914721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.914721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:38.914721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.914721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:38.915042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.915042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:38.915042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:38.915042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:38.915042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:38.915042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:38.915042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:38.915042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:00:38.915042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:38.915042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:38.915042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.915042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:38.915646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:38.916055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.916055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:38.916055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.916055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:38.916296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:38.916961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:38.917639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.917639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:38.917639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:38.917639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.917639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:38.917639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.917639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:38.917639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:38.917639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:38.917639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:38.917990 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:38.918038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:38.918369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.919003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:38.919003 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.919003 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:38.919003 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:38.919003 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.919003 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:38.919003 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.919003 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:38.919802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:38.920264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:38.920264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:38.920264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.920264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:38.920264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:38.920264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.920264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:38.920264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.920264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:38.920680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.920680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:38.920680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:38.920680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:38.920680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:38.920680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:38.920680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:38.920680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:38.920680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:38.920680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.920680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:38.921021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:38.921481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.921481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:38.921481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.921481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:38.921797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:38.922392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:38.922920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.922920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:38.922920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.922920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:38.922920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:38.922920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:38.923162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:38.923162 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:38.924049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:38.924049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:38.924049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.924567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:38.924567 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.924567 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:38.924567 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:38.924567 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.924567 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:38.924567 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.924567 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:38.925513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.925513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:38.925513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:38.925513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:38.925513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:38.925513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:38.925513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:38.925513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.925513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:38.925513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:38.925513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:38.925513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.925921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:38.925921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:38.925921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:38.925921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:38.925921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:38.925921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:38.925921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:38.925921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:38.925921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.925921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:38.925921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:38.925921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.925921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:38.926338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.926338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:38.926487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:38.927000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:38.927587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:38.927587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:38.927587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:41.206693 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:41.207912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:41.207912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:41.209299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:41.258313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:41.258589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:41.260339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:41.260339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:41.260339 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:41.260339 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:41.260339 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:41.260339 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:41.260339 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:41.260339 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:41.260857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:41.260857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:41.260857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:41.260857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:41.260857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:41.260857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:41.260857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:41.260857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:41.260857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:41.260857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:41.260857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:41.260857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:41.261444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:41.261444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:41.261444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:41.261444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:41.261444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:41.261444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:41.261444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:41.261444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:41.261444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:41.261444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:41.261444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:41.261444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:41.261444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:41.262088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:41.262968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:41.263957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:41.263957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:41.263957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:41.263957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:41.263957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:41.263957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:43.903715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:43.904722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:43.904855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:43.906259 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:43.906662 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:43.906907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.907841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:43.907841 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.907841 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:43.907841 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:43.907841 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.907841 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:43.907841 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.907841 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:43.909535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.909535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:43.909535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:43.909535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:43.909535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:43.909535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:43.909535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:43.909535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.909535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:43.909535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:43.909535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:43.909535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.910138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:43.910138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:43.910138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:43.910138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:43.910138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:43.910138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:43.910138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:43.910138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:43.910138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.910138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:43.910138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:43.910138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.910138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:43.910692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.910692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:43.911387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:43.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.914667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:43.914667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.914667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:43.914667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:43.914667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:43.914667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:43.914667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:00:43.914667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:43.915084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.915574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:43.916013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.916013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:43.916013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:43.916013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:43.916013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:43.916013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:43.916013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:43.916013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:00:43.916013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:43.916013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:43.916013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.916013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:43.916938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:43.917720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:43.918496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:43.919337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:43.919699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:43.919699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:43.919699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:43.919699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.919829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:43.919829 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.919829 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:43.919829 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:43.919829 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.919829 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:43.919829 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.919829 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:43.920634 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:43.921016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:43.921016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:43.921016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.921016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:43.921016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:43.921016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.921016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:43.921016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.921016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:43.921566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.921566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:43.921566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:43.921566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:43.921566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:43.921566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:43.921566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:43.921566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:43.921566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:43.921566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.921566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:43.922021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:43.922375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.922375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:43.922375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.922375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:43.922726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:43.923058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:43.923414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.923414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:43.923414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.923414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:43.923414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:43.923414 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:43.923540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:43.923858 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:43.924117 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:43.924521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:43.924739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.925482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:43.925482 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.925482 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:43.925482 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:43.925482 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.925482 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:43.925482 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.925482 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:43.926399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.926399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:43.926399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:43.926399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:43.926399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:43.926399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:43.926399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:43.926399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.926399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:43.926399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:43.926399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:43.926399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.926897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:43.926897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:43.926897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:43.926897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:43.926897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:43.926897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:43.926897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:43.926897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:43.926897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.926897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:43.926897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:43.926897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.926897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:43.927404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:43.927735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:43.928059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:43.928059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:43.928059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:43.928059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:43.928059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:43.928059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:45.773179 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:45.774520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:45.774520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:45.784725 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 07 16:00:45.784852 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105503) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 07 16:00:45.784998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 16:00:45.825071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, resource={'id': '8b456c85-5e2d-47f8-9886-ebf65932cf32', 'name': 'private', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4a00c537-ed4f-405c-b241-82e7bc2cf3fa', '9fc89727-343d-403a-acd7-db6b4950e7b1'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:25:31Z', 'updated_at': '2026-05-07T15:25:34Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 42799}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:45.825623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:45.826338 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-b0f8152d-626b-440e-9383-919ec905ab38, external_ids_update=None, columns={'tag': 42799}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:45.826550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.828038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:45.828038 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:45.828038 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:45.828038 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:45.828038 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:45.828038 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:45.828038 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.828038 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:45.829088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.829088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:45.829088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 16:00:45.829088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 16:00:45.829088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 16:00:45.829088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 16:00:45.829088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 16:00:45.829088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:45.829088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:45.829088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:45.829088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:45.829521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:45.829998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:45.829998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.829998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:45.829998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 16:00:45.867459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, resource={'id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'name': 'public', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6aa26064-0a62-428b-a8ed-fef52fc73984', 'adf79775-4dfa-4110-bca0-59be9560299c'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-07T15:25:37Z', 'updated_at': '2026-05-07T15:25:45Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:45.867951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:45.868492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-2b8b3cbc-130c-4894-94e1-ab0729ec7077, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:45.868751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.869616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:45.869616 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:45.869616 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:45.869616 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:45.869616 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:45.869616 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:45.869616 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.869616 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:45.870738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.870738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:45.870738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 16:00:45.870738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 16:00:45.870738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 16:00:45.870738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 16:00:45.870738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 16:00:45.870738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:45.870738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:45.870738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:45.870738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:45.871685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:45.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:45.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:45.872323 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 16:00:45.901337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, resource={'id': '71aea053-d575-4d13-afc5-e6241ad6a15f', 'name': 'admin_net', 'tenant_id': '74d9c701d5cb4de6a15233057b01e413', 'project_id': '74d9c701d5cb4de6a15233057b01e413', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8e8a2cf7-97a7-4d62-ae4a-fa6c2139e811'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:30:51Z', 'updated_at': '2026-05-07T15:30:52Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 51150}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:45.902038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:45.902784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d5042c49-4c90-465a-b356-c18cf966a58b, external_ids_update=None, columns={'tag': 51150}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:45.903046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.904243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:45.904243 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:45.904243 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:45.904243 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:45.904243 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:45.904243 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:45.904243 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.904243 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:45.906625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.906625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:45.906625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 16:00:45.906625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 16:00:45.906625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 16:00:45.906625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 16:00:45.906625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 16:00:45.906625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:45.906625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:45.906625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:45.906625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:45.906962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:45.907396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:45.907396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.907396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:45.907396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 16:00:45.936833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, resource={'id': '4fb571ad-ed2f-4edf-a199-cb52ff142698', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'tenant_id': '91e1e016e0f8434890282eb48ed58760', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2e6720c7-3159-41b9-80a2-fa6cf6a18808'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:31:26Z', 'updated_at': '2026-05-07T15:31:27Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 29129}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:45.937843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:45.938537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13b8a284-1038-44ba-86e4-667f771dfae2, external_ids_update=None, columns={'tag': 29129}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:45.938725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 16:00:45.941203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 16:00:45.941673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 16:00:45.941673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:45.941673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:45.941673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:45.941673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:45.941673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:45.941673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:45.941673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:45.941673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:45.941673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:45.941673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:45.941673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:45.941673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:45.942067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:45.942067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:45.942067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:45.942067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:45.942067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:45.942067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:45.942067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:45.942067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:45.942067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:45.942067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 16:00:46.035977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, resource={'id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'enable_snat': True}, 'gw_port_id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-jseci', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'ip_address': '172.24.5.213', 'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984'}, {'ip_address': '2001:db8::15a', 'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 3, 'tenant_id': '91e1e016e0f8434890282eb48ed58760'}, resource_type=routers, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:46.036161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:46.036317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:46.036454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:46.036563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.037303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:46.037303 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:46.037303 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:46.037303 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:46.037303 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:46.037303 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:46.037303 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.037303 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:46.037702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Unable to update router 0938bfb8-1052-49e2-bd69-e5fda45e8a2f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.039237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.039237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:46.039237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 16:00:46.039237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 16:00:46.039237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 16:00:46.039237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 16:00:46.039237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 07 16:00:46.039237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 07 16:00:46.039237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 07 16:00:46.039237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 07 16:00:46.039237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:46.039870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:46.040535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:46.040535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:46.040535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:46.040535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:46.040535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:46.040535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:46.040535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:46.040535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.040535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:46.040535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 16:00:46.153026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, resource={'id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'name': '', 'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'project_id': '', 'mac_address': 'fa:16:3e:f6:db:65', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000172501', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:46.153330 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '6aa26064-0a62-428b-a8ed-fef52fc73984 adf79775-4dfa-4110-bca0-59be9560299c', 'neutron:network_name': 'neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.213/24', '2001:db8::15a/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:46.153646 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=b30ffafe-8df3-4064-bcbf-e3d1daad619c, lrouter_port=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:46.153920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.154953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:46.154953 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:46.154953 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:46.154953 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:46.154953 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:46.154953 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:46.154953 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.154953 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:46.155853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.155853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:46.155853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 16:00:46.155853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 16:00:46.155853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 16:00:46.155853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 16:00:46.155853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 07 16:00:46.155853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:46.155853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:46.155853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:46.155853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:46.156447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:46.157082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:46.157082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.157082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:46.157082 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_for_inconsistencies finished in 0.382 seconds
May 07 16:00:46.207337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:46.207905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:46.208095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:46.209409 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:46.243352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:46.243654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.244287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:46.244287 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:46.244287 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:46.244287 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:46.244287 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:46.244287 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:46.244287 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.244287 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:46.245039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.245039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:46.245039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:46.245039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:46.245039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:46.245039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:46.245039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:46.245039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:46.245039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:46.245039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:46.245039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:46.245039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:46.245979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:46.245979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:46.245979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:46.245979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:46.245979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:46.245979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:46.245979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:46.245979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:46.245979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:46.245979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:46.245979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:46.245979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:46.245979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:46.246878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:46.247970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:46.249354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:46.249354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:46.249354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:46.249354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:46.249354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:46.249354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:48.905789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:48.906509 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:48.906708 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:48.907855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:48.908170 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:48.908440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.909307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:48.909307 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.909307 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:48.909307 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:48.909307 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.909307 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:48.909307 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.909307 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:48.910851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.910851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:48.910851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:48.910851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:48.910851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:48.910851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:48.910851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:48.910851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.910851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:48.910851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:48.910851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:48.910851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.911230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:48.911230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:48.911230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:48.911230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:48.911230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:48.911230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:48.911230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:48.911230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:48.911230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.911230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:48.911230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:48.911230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.911230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:48.911595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.911595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:48.912803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:48.913260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.913789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:48.913789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.913789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:48.913789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:48.913789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:48.913789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:48.913991 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:00:48.914371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:48.914685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.915438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:48.915438 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.915438 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:48.915438 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:48.915438 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.915438 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:48.915438 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.915438 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:48.916611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:48.917049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:48.917049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:48.917049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.917049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:48.917049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:48.917049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.917049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:48.917049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.917049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:48.917901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.917901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:48.917901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:48.917901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:48.917901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:48.917901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:48.917901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:48.917901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:00:48.917901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:48.917901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:48.917901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.917901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:48.918286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:48.918616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.918616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:48.918616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.918616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:48.918616 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:48.919821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:48.920299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:48.920788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.920788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:48.920788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:48.920788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.920788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:48.920788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.920788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:48.920788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:48.920788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:48.921038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:48.921346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:48.921692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.922436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:48.922436 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.922436 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:48.922436 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:48.922436 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.922436 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:48.922436 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.922436 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:48.923192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:48.923642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:48.923642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:48.923642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.923642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:48.923642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:48.923642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.923642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:48.923642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.923642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:48.924363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.924363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:48.924363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:48.924363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:48.924363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:48.924363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:48.924363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:48.924363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:48.924363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:48.924363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.924363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:48.924695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:48.925064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.925064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:48.925064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.925064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:48.925064 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:48.925830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:48.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:48.926855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.926855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:48.926855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.926855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:48.926855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:48.926855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:48.926855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:48.926855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:48.927105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:48.927105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.927501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:48.927501 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.927501 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:48.927501 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:48.927501 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.927501 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:48.927501 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.927501 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:48.928084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.928084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:48.928084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:48.928084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:48.928084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:48.928084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:48.928084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:48.928084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.928084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:48.928084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:48.928084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:48.928084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.928540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:48.928540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:48.928540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:48.928540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:48.928540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:48.928540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:48.928540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:48.928540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:48.928540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.928540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:48.928540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:48.928540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.928540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:48.928874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:48.929319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:48.929794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:48.929794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:48.929794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:48.929794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:48.929794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:48.929794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:51.207939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:51.208552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:51.208756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:51.210137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:51.250552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:51.250864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:51.251493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:51.251493 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:51.251493 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:51.251493 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:51.251493 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:51.251493 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:51.251493 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:51.251493 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:51.252183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:51.252183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:51.252183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:51.252183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:51.252183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:51.252183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:51.252183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:51.252183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:51.252183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:51.252183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:51.252183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:51.252183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:51.252876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:51.252876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:51.252876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:51.252876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:51.252876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:51.252876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:51.252876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:51.252876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:51.252876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:51.252876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:51.252876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:51.252876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:51.252876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:51.253532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:51.254336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:51.255248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:51.255248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:51.255248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:51.255248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:51.255248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:51.255248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:53.906617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:53.907280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:53.907406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:53.908408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:53.908610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:53.908735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.909337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:53.909337 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.909337 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:53.909337 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:53.909337 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.909337 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:53.909337 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.909337 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:53.910783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.910783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:53.910783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:53.910783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:53.910783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:53.910783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:53.910783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:53.910783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.910783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:53.910783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:53.910783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:53.910783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.911199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:53.911199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:53.911199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:53.911199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:53.911199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:53.911199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:53.911199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:53.911199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:53.911199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.911199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:53.911199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:53.911199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.911199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:53.911612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.911612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:53.911885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:53.912391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.912913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:53.912913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.912913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:53.913561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:53.913835 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:53.913946 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:53.914065 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:00:53.914562 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:53.914731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.915191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:53.915191 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.915191 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:53.915191 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:53.915191 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.915191 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:53.915191 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.915191 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:53.915962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:53.916512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:53.916512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:53.916512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.916512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:53.916512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:53.916512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.916512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:53.916512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.916512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:53.916816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.916816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:53.916816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:53.916816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:53.916816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:53.916816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:53.916816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:53.916816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:00:53.916816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:53.916816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:53.916816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.916816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:53.917233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:53.917659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.917659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:53.917659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.917659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:53.917884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:53.918426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:53.918946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.918946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:53.918946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:53.918946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.918946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:53.918946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.918946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:53.920179 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:53.920482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:53.920594 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:53.920726 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:53.921272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:53.921489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.921954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:53.921954 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.921954 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:53.921954 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:53.921954 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.921954 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:53.921954 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.921954 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:53.922736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:53.923230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:53.923230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:53.923230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.923230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:53.923230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:53.923230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.923230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:53.923230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.923230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:53.923647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.923647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:53.923647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:53.923647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:53.923647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:53.923647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:53.923647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:53.923647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:53.923647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:53.923647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.923647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:53.924006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:53.924430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.924430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:53.924430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.924430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:53.924780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:53.925300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:53.925833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.925833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:53.925833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.925833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:53.926149 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:53.926451 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:53.926550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:53.926715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:53.927136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:53.927386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:53.927496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.927962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:53.927962 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.927962 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:53.927962 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:53.927962 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.927962 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:53.927962 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.927962 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:53.928808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.928808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:53.928808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:53.928808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:53.928808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:53.928808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:53.928808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:53.928808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.928808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:53.928808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:53.928808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:53.928808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.929240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:53.929240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:53.929240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:53.929240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:53.929240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:53.929240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:53.929240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:53.929240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:53.929240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.929240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:53.929240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:53.929240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.929240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:53.929626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.929626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:53.929816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:53.930337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:53.930921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:53.930921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:53.930921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:56.208616 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:56.209425 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:56.209550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:56.210671 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:56.252434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:56.252847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:56.253836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:56.253836 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:56.253836 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:56.253836 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:56.253836 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:56.253836 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:56.253836 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:56.253836 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:56.255282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:56.255282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:56.255282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:56.255282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:56.255282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:56.255282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:56.255282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:56.255282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:56.255282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:56.255282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:56.255282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:56.255282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:56.255977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:56.255977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:56.255977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:56.255977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:56.255977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:56.255977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:56.255977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:56.255977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:56.255977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:56.255977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:56.255977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:56.255977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:56.255977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:56.256654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:56.257380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:56.258193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:56.258193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:56.258193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:56.258193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:56.258193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:56.258193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:58.907193 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:58.907950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:58.908258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:58.909163 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:58.909372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:58.909533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.910374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:58.910374 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.910374 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:58.910374 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:58.910374 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.910374 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:58.910374 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.910374 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:58.911968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.911968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:58.911968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:58.911968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:58.911968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:58.911968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:58.911968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:58.911968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.911968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:58.911968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:58.911968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:58.911968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.912546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:58.912546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:58.912546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:58.912546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:58.912546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:58.912546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:58.912546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:58.912546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:58.912546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.912546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:58.912546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:58.912546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.912546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:58.913104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.913104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:58.913696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:58.914454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.915282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:58.915282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.915282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:58.915282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:58.915282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:58.915282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:58.915282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:00:58.915282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:58.916069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.916355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:58.916355 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.916355 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:58.916355 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:58.916355 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.916355 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:58.916355 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.916355 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:58.917195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:58.917646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:58.917646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:58.917646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.917646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:58.917646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:58.917646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.917646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:58.917646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.917646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:58.918267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.918267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:58.918267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:58.918267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:58.918267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:58.918267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:58.918267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:58.918267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:00:58.918267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:58.918267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:58.918267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.918267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:58.918635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:58.919024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.919024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:58.919024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.919024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:58.919563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:58.920034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:58.920564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.920564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:58.920564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:58.920564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.920564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:58.920564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.920564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:58.920564 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:58.920880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:58.921026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:58.921189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:58.922217 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:58.922481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.922912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:58.922912 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.922912 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:58.922912 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:58.922912 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.922912 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:58.922912 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.922912 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:00:58.923727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:00:58.924294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:58.924294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:00:58.924294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.924294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:00:58.924294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:00:58.924294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.924294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:00:58.924294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.924294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:00:58.924832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.924832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:58.924832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:58.924832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:58.924832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:58.924832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:58.924832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:58.924832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:58.924832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:58.924832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.924832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:58.925174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:58.925587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.925587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:58.925587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.925587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:58.926812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:58.927314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:58.927851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.927851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:58.927851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.927851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:00:58.927851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:00:58.927851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:00:58.927851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:00:58.927851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:00:58.928110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:58.928320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:00:58.928455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.929110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:00:58.929110 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.929110 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:00:58.929110 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:00:58.929110 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.929110 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:00:58.929110 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.929110 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:00:58.929984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.929984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:00:58.929984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:58.929984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:00:58.929984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:00:58.929984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:58.929984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:58.929984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.929984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:58.929984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:58.929984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:00:58.929984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.930389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:00:58.930389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:58.930389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:00:58.930389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:58.930389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:00:58.930389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:00:58.930389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:58.930389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:00:58.930389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.930389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:00:58.930389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:00:58.930389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.930389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:00:58.930747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.930747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:00:58.931222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:00:58.931700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:00:58.932272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:00:58.932272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:00:58.932272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:01.209341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:01.210054 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:01.210237 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:01.211631 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:01.250969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:01.251312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:01.252405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:01.252405 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:01.252405 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:01.252405 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:01.252405 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:01.252405 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:01.252405 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:01.252405 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:01.253944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:01.253944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:01.253944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:01.253944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:01.253944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:01.253944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:01.253944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:01.253944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:01.253944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:01.253944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:01.253944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:01.253944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:01.254618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:01.254618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:01.254618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:01.254618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:01.254618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:01.254618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:01.254618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:01.254618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:01.254618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:01.254618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:01.254618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:01.254618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:01.254618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:01.255110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:01.255110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:01.255403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:01.255875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:01.256447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:01.256447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:01.256447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:03.907928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:03.908909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:03.908909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:03.909900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:03.910280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:03.910492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.911308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:03.911308 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.911308 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:03.911308 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:03.911308 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.911308 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:03.911308 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.911308 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:03.912690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.912690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:03.912690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:03.912690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:03.912690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:03.912690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:03.912690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:03.912690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.912690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:03.912690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:03.912690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:03.912690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.913545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:03.913545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:03.913545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:03.913545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:03.913545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:03.913545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:03.913545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:03.913545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:03.913545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.913545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:03.913545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:03.913545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.913545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:03.914329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.914329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:03.914462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:03.915144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.915751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:03.915751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.915751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:03.915751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:03.915751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:03.915751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:03.915751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:01:03.916006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:03.916116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.916572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:03.916572 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.916572 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:03.916572 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:03.916572 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.916572 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:03.916572 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.916572 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:03.917320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:03.917806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:03.917806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:03.917806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.917806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:03.917806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:03.917806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.917806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:03.917806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.917806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:03.918193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.918193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:03.918193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:03.918193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:03.918193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:03.918193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:03.918193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:03.918193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:01:03.918193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:03.918193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:03.918193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.918193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:03.918586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:03.918953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.918953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:03.918953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.918953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:03.919272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:03.919718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:03.920240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.920240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:03.920240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:03.920240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.920240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:03.920240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.920240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:03.920951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:03.921220 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:03.921381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:03.921514 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:03.921791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:03.922065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.922549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:03.922549 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.922549 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:03.922549 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:03.922549 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.922549 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:03.922549 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.922549 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:03.923298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:03.923674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:03.923674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:03.923674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.923674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:03.923674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:03.923674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.923674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:03.923674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.923674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:03.924184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.924184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:03.924184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:03.924184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:03.924184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:03.924184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:03.924184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:03.924184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:03.924184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:03.924184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.924184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:03.924530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:03.924882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.924882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:03.924882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.924882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:03.925338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:03.925828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:03.926379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.926379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:03.926379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.926379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:03.927427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:03.927688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:03.927806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:03.927983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:03.928268 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:03.928521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:03.928673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.929134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:03.929134 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.929134 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:03.929134 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:03.929134 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.929134 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:03.929134 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.929134 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:03.930022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.930022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:03.930022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:03.930022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:03.930022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:03.930022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:03.930022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:03.930022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.930022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:03.930022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:03.930022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:03.930022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.930422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:03.930422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:03.930422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:03.930422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:03.930422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:03.930422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:03.930422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:03.930422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:03.930422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.930422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:03.930422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:03.930422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.930422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:03.930748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.930748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:03.931051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:03.931515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:03.932023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:03.932023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:03.932023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:06.210108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:06.211003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:06.211003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:06.211929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:06.242039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:06.242226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:06.242945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:06.242945 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:06.242945 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:06.242945 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:06.242945 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:06.242945 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:06.242945 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:06.242945 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:06.243971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:06.243971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:06.243971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:06.243971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:06.243971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:06.243971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:06.243971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:06.243971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:06.243971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:06.243971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:06.243971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:06.243971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:06.244566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:06.244566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:06.244566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:06.244566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:06.244566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:06.244566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:06.244566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:06.244566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:06.244566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:06.244566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:06.244566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:06.244566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:06.244566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:06.245101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:06.245817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:06.246921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:06.246921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:06.246921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:06.246921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:06.246921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:06.246921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:08.908563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:08.909637 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:08.909637 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:08.910153 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:08.910445 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:08.910848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.911270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:08.911270 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.911270 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:08.911270 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:08.911270 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.911270 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:08.911270 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.911270 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:08.912285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.912285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:08.912285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:08.912285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:08.912285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:08.912285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:08.912285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:08.912285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.912285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:08.912285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:08.912285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:08.912285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.913006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:08.913006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:08.913006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:08.913006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:08.913006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:08.913006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:08.913006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:08.913006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:08.913006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.913006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:08.913006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:08.913006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.913006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:08.913804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:08.914428 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.915389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:08.915389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:08.915389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.915389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:08.915389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.915389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:08.915389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:08.915647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:08.915647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:08.915647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:01:08.916124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:08.916446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.917153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:08.917153 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.917153 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:08.917153 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:08.917153 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.917153 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:08.917153 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.917153 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:08.918005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:08.918533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:08.918533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:08.918533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.918533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:08.918533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:08.918533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.918533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:08.918533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.918533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:08.918848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.918848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:08.918848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:08.918848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:08.918848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:08.918848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:08.918848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:08.918848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:01:08.918848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:08.918848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:08.918848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.918848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:08.919200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:08.919608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.919608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:08.919608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.919608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:08.919794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:08.920280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:08.920799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.920799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:08.920799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:08.920799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.920799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:08.920799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.920799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:08.921289 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:08.921522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:08.921625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:08.921749 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:08.922383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:08.922620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.923300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:08.923300 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.923300 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:08.923300 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:08.923300 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.923300 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:08.923300 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.923300 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:08.924028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:08.924460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:08.924460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:08.924460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.924460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:08.924460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:08.924460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.924460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:08.924460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.924460 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:08.924883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.924883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:08.924883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:08.924883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:08.924883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:08.924883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:08.924883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:08.924883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:08.924883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:08.924883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.924883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:08.925232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:08.925667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.925667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:08.925667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.925667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:08.925934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:08.926445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:08.926951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.926951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:08.926951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.926951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:08.927713 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:08.927939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:08.928048 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:08.928221 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:08.928587 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:08.928819 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:08.928966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.930360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:08.930360 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.930360 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:08.930360 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:08.930360 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.930360 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:08.930360 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.930360 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:08.932374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.932374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:08.932374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:08.932374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:08.932374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:08.932374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:08.932374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:08.932374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.932374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:08.932374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:08.932374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:08.932374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.932771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:08.932771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:08.932771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:08.932771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:08.932771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:08.932771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:08.932771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:08.932771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:08.932771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.932771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:08.932771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:08.932771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.932771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:08.933124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.933124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:08.934108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:08.934582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:08.935088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:08.935088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:08.935088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:11.210909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:11.211940 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:11.211940 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:11.212802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:11.220367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=433 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 16:01:11.247919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:11.248280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:11.249169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:11.249169 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:11.249169 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:11.249169 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:11.249169 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:11.249169 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:11.249169 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:11.249169 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:11.249968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:11.249968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:11.249968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:11.249968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:11.249968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:11.249968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:11.249968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:11.249968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:11.249968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:11.249968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:11.249968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:11.249968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:11.250548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:11.250548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:11.250548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:11.250548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:11.250548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:11.250548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:11.250548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:11.250548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:11.250548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:11.250548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:11.250548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:11.250548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:11.250548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:11.251133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:11.251810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:11.252802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:11.252802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:11.252802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:11.252802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:11.252802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:11.252802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:13.909122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:13.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:13.909698 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:13.910512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:13.910789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:13.911037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.911949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:13.911949 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.911949 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:13.911949 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:13.911949 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.911949 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:13.911949 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.911949 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:13.912861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.912861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:13.912861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:13.912861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:13.912861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:13.912861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:13.912861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:13.912861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.912861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:13.912861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:13.912861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:13.912861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.913229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:13.913229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:13.913229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:13.913229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:13.913229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:13.913229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:13.913229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:13.913229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:13.913229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.913229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:13.913229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:13.913229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.913229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:13.913532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.913532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:13.913865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:13.914306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.914798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:13.914798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.914798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:13.915578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:13.915816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:13.915922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:13.916037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:01:13.916363 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:13.916536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.917009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:13.917009 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.917009 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:13.917009 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:13.917009 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.917009 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:13.917009 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.917009 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:13.917705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:13.918110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:13.918110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:13.918110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.918110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:13.918110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:13.918110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.918110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:13.918110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.918110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:13.918509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.918509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:13.918509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:13.918509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:13.918509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:13.918509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:13.918509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:13.918509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:01:13.918509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:13.918509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:13.918509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.918509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:13.918880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:13.919263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.919263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:13.919263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.919263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:13.919481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:13.919897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:13.920380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.920380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:13.920380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:13.920380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.920380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:13.920380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.920380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:13.921623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:13.921855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:13.921965 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:13.922163 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:13.922475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:13.922628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.923030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:13.923030 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.923030 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:13.923030 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:13.923030 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.923030 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:13.923030 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.923030 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:13.923726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:13.924127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:13.924127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:13.924127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.924127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:13.924127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:13.924127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.924127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:13.924127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.924127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:13.924564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.924564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:13.924564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:13.924564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:13.924564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:13.924564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:13.924564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:13.924564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:13.924564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:13.924564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.924564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:13.924877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:13.925281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.925281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:13.925281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.925281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:13.925644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:13.926109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:13.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:13.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.926624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:13.928189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:13.928495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:13.928596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:13.928736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:13.929108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:13.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:13.929853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.930515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:13.930515 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.930515 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:13.930515 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:13.930515 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.930515 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:13.930515 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.930515 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:13.931304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.931304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:13.931304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:13.931304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:13.931304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:13.931304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:13.931304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:13.931304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.931304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:13.931304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:13.931304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:13.931304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.931672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:13.931672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:13.931672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:13.931672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:13.931672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:13.931672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:13.931672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:13.931672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:13.931672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.931672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:13.931672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:13.931672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.931672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:13.932019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.932019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:13.932263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:13.932679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:13.933176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:13.933176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:13.933176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:16.211673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:16.212427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:16.212427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:16.213825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:16.247136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:16.247551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:16.248390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:16.248390 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:16.248390 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:16.248390 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:16.248390 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:16.248390 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:16.248390 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:16.248390 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:16.249179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:16.249179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:16.249179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:16.249179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:16.249179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:16.249179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:16.249179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:16.249179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:16.249179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:16.249179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:16.249179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:16.249179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:16.249821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:16.249821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:16.249821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:16.249821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:16.249821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:16.249821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:16.249821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:16.249821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:16.249821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:16.249821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:16.249821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:16.249821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:16.249821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:16.250405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:16.251324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:16.252451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:16.252451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:16.252451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:16.252451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:16.252451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:16.252451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:18.909902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:18.910654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:18.910654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:18.911314 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:18.911625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:18.911851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.912988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:18.912988 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.912988 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:18.912988 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:18.912988 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.912988 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:18.912988 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.912988 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:18.914646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.914646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:18.914646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:18.914646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:18.914646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:18.914646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:18.914646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:18.914646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.914646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:18.914646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:18.914646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:18.914646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.915264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:18.915264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:18.915264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:18.915264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:18.915264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:18.915264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:18.915264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:18.915264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:18.915264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.915264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:18.915264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:18.915264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.915264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:18.915868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.915868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:18.916164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:18.917327 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.918080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:18.918080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.918080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:18.918080 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:18.918080 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:18.918080 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:18.918080 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:01:18.918359 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:18.919338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.919338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:18.919338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.919338 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:18.919338 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:18.919338 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.919338 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:18.919338 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.919338 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:18.920485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:18.920983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:18.920983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:18.920983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.920983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:18.920983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:18.920983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.920983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:18.920983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.920983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:18.921334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.921334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:18.921334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:18.921334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:18.921334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:18.921334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:18.921334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:18.921334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:01:18.921334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:18.921334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:18.921334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.921334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:18.921699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:18.922492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:18.922917 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:18.923719 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:18.924262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:18.924262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:18.924262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:18.924381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.925122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:18.925122 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.925122 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:18.925122 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:18.925122 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.925122 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:18.925122 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.925122 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:18.926025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:18.926439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:18.926439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:18.926439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.926439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:18.926439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:18.926439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.926439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:18.926439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.926439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:18.926944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.926944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:18.926944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:18.926944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:18.926944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:18.926944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:18.926944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:18.926944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:18.926944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:18.926944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.926944 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:18.927287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:18.927680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.927680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:18.927680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.927680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:18.928303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:18.928765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:18.929277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.929277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:18.929277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.929277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:18.929277 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:18.929277 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:18.929277 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:18.929277 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:18.929794 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:18.930303 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:18.930577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.931683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:18.931683 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.931683 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:18.931683 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:18.931683 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.931683 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:18.931683 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.931683 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:18.932691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.932691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:18.932691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:18.932691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:18.932691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:18.932691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:18.932691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:18.932691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.932691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:18.932691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:18.932691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:18.932691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.933064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:18.933064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:18.933064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:18.933064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:18.933064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:18.933064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:18.933064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:18.933064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:18.933064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.933064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:18.933064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:18.933064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.933064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:18.933448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.933448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:18.933687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:18.934125 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:18.934692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:18.934692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:18.934692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:21.212132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:21.212755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:21.212942 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:21.214199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:21.247826 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:21.248232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:21.249038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:21.249038 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:21.249038 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:21.249038 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:21.249038 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:21.249038 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:21.249038 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:21.249038 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:21.249813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:21.249813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:21.249813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:21.249813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:21.249813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:21.249813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:21.249813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:21.249813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:21.249813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:21.249813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:21.249813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:21.249813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:21.250468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:21.250468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:21.250468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:21.250468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:21.250468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:21.250468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:21.250468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:21.250468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:21.250468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:21.250468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:21.250468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:21.250468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:21.250468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:21.251059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:21.251823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:21.252717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:21.252717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:21.252717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:21.252717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:21.252717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:21.252717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:23.910481 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:23.911297 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:23.911386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:23.912491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:23.912872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:23.913130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.913982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:23.913982 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.913982 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:23.913982 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:23.913982 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.913982 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:23.913982 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.913982 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:23.915440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.915440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:23.915440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:23.915440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:23.915440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:23.915440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:23.915440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:23.915440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.915440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:23.915440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:23.915440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:23.915440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.916142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:23.916142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:23.916142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:23.916142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:23.916142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:23.916142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:23.916142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:23.916142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:23.916142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.916142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:23.916142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:23.916142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.916142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:23.916916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.916916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:23.917590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:23.918517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.919102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:23.919102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.919102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:23.919102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:23.919102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:23.919102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:23.919102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:01:23.919102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:23.919399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.919704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:23.919704 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.919704 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:23.919704 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:23.919704 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.919704 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:23.919704 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.919704 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:23.920602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:23.921091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:23.921091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:23.921091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.921091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:23.921091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:23.921091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.921091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:23.921091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.921091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:23.921472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.921472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:23.921472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:23.921472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:23.921472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:23.921472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:23.921472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:23.921472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:01:23.921472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:23.921472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:23.921472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.921472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:23.921912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:23.922316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.922316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:23.922316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.922316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:23.922556 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:23.923017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:23.923564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.923564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:23.923564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:23.923564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.923564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:23.923564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.923564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:23.923564 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:23.923797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:23.923797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:23.923976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:23.924311 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:23.924587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.925074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:23.925074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.925074 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:23.925074 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:23.925074 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.925074 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:23.925074 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.925074 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:23.925879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:23.926314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:23.926314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:23.926314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.926314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:23.926314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:23.926314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.926314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:23.926314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.926314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:23.926875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.926875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:23.926875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:23.926875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:23.926875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:23.926875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:23.926875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:23.926875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:23.926875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:23.926875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.926875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:23.929257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:23.929624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:23.930101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:23.930610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:23.930610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:23.930610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:23.930610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:23.930610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.930610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:23.930610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:23.930610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.930610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:23.930610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.930610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:23.930610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:23.930610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:23.930994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:23.930994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:23.930994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:23.931158 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:23.931317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.931882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:23.931882 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.931882 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:23.931882 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:23.931882 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.931882 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:23.931882 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.931882 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:23.933227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.933227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:23.933227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:23.933227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:23.933227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:23.933227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:23.933227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:23.933227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.933227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:23.933227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:23.933227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:23.933227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.933643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:23.933643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:23.933643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:23.933643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:23.933643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:23.933643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:23.933643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:23.933643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:23.933643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.933643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:23.933643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:23.933643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.933643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:23.934015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.934015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:23.934358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:23.934839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:23.935429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:23.935429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:23.935429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:26.212846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:26.213724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:26.213724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:26.214335 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:26.251188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:26.252298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:26.252544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:26.252544 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:26.252544 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:26.252544 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:26.252544 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:26.252544 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:26.252544 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:26.252544 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:26.253474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:26.253474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:26.253474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:26.253474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:26.253474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:26.253474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:26.253474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:26.253474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:26.253474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:26.253474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:26.253474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:26.253474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:26.255379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:26.255379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:26.255379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:26.255379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:26.255379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:26.255379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:26.255379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:26.255379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:26.255379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:26.255379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:26.255379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:26.255379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:26.255379 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:26.256113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:26.256620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:26.257183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:26.257183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:26.257183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:26.257183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:26.257183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:26.257183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:28.911025 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:28.911891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:28.911891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:28.913149 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:28.913491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:28.914234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.914572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:28.914572 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.914572 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:28.914572 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:28.914572 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.914572 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:28.914572 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.914572 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:28.915558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.915558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:28.915558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:28.915558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:28.915558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:28.915558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:28.915558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:28.915558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.915558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:28.915558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:28.915558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:28.915558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.915990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:28.915990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:28.915990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:28.915990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:28.915990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:28.915990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:28.915990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:28.915990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:28.915990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.915990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:28.915990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:28.915990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.915990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:28.916334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.916334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:28.917322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:28.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.920716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:28.920716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.920716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:28.920716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:28.920716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:28.920716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:28.920716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:01:28.920716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:28.920992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.920992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:28.920992 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.920992 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:28.920992 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:28.920992 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.920992 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:28.920992 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.920992 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:28.922438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:28.923488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:28.923488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:28.923488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.923488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:28.923488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:28.923488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.923488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:28.923488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.923488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:28.924000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.924000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:28.924000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:28.924000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:28.924000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:28.924000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:28.924000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:28.924000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:01:28.924000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:28.924000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:28.924000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.924000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:28.924670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:28.925332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.925332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:28.925332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.925332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:28.925332 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:28.925775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:28.926611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:28.927468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.927468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:28.927468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:28.927468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.927468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:28.927468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.927468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:28.927468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:28.927468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:28.927468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:28.927951 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:28.927951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.927951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:28.927951 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.927951 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:28.927951 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:28.927951 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.927951 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:28.927951 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.927951 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:28.928833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:28.929524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:28.929524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:28.929524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.929524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:28.929524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:28.929524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.929524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:28.929524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.929524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:28.930052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.930052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:28.930052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:28.930052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:28.930052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:28.930052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:28.930052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:28.930052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:28.930052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:28.930052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.930052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:28.930644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:28.931288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.931288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:28.931288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.931288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:28.931730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:28.932636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:28.933720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.933720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:28.933720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.933720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:28.933720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:28.933720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:28.933720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:28.933720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:28.934288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:28.934288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:28.934288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.934288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:28.934288 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.934288 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:28.934288 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:28.934288 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.934288 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:28.934288 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.934288 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:28.934748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.934748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:28.934748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:28.934748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:28.934748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:28.934748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:28.934748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:28.934748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.934748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:28.934748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:28.934748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:28.934748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.935026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:28.935026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:28.935026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:28.935026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:28.935026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:28.935026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:28.935026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:28.935026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:28.935026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.935026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:28.935026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:28.935026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.935026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:28.935310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.935310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:28.935403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:28.935719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:28.936110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:28.936110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:28.936110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:31.213625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:31.214265 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:31.214441 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:31.215695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:31.251719 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:31.251971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:31.252673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:31.252673 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:31.252673 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:31.252673 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:31.252673 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:31.252673 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:31.252673 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:31.252673 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:31.253676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:31.253676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:31.253676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:31.253676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:31.253676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:31.253676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:31.253676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:31.253676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:31.253676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:31.253676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:31.253676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:31.253676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:31.254243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:31.254243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:31.254243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:31.254243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:31.254243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:31.254243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:31.254243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:31.254243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:31.254243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:31.254243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:31.254243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:31.254243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:31.254243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:31.254829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:31.255516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:31.256374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:31.256374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:31.256374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:31.256374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:31.256374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:31.256374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:33.911722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:33.912552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:33.912626 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:33.913550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:33.913723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:33.913841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.914488 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:33.914488 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.914488 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:33.914488 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:33.914488 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.914488 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:33.914488 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.914488 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:33.916310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.916310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:33.916310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:33.916310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:33.916310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:33.916310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:33.916310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:33.916310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.916310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:33.916310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:33.916310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:33.916310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.916930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:33.916930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:33.916930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:33.916930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:33.916930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:33.916930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:33.916930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:33.916930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:33.916930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.916930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:33.916930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:33.916930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.916930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:33.917568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.917568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:33.918544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:33.919279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.920041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:33.920041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.920041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:33.946508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:33.946508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:33.946508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:33.946508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:33.946508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:33.946508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:01:33.947357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:33.947357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.947506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:33.947506 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.947506 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:33.947506 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:33.947506 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.947506 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:33.947506 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.947506 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:33.948451 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:33.949180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:33.949180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:33.949180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.949180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:33.949180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:33.949180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.949180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:33.949180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.949180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:33.949616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.949616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:33.949616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:33.949616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:33.949616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:33.949616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:33.949616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:33.949616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:01:33.949616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:33.949616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:33.949616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.949616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:33.950157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:33.950811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.950811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:33.950811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.950811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:33.951028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:33.951897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:33.952716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.952716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:33.952716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:33.952716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.952716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:33.952716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.952716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:33.952716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:33.952716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:33.952716 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:33.953231 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:33.953231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.953231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:33.953231 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.953231 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:33.953231 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:33.953231 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.953231 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:33.953231 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.953231 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:33.954069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:33.954779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:33.954779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:33.954779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.954779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:33.954779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:33.954779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.954779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:33.954779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.954779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:33.955167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.955167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:33.955167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:33.955167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:33.955167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:33.955167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:33.955167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:33.955167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:33.955167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:33.955167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.955167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:33.955730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:33.956373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:33.957135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:33.957871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:33.957871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:33.957871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:33.957871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:33.957871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.957871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:33.957871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:33.957871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.957871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:33.957871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.957871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:33.957871 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:33.957871 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:33.958313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:33.958313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:33.958313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:33.958313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.958313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:33.958313 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.958313 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:33.958313 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:33.958313 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.958313 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:33.958313 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.958313 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:33.958831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.958831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:33.958831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:33.958831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:33.958831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:33.958831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:33.958831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:33.958831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.958831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:33.958831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:33.958831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:33.958831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.959194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:33.959194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:33.959194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:33.959194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:33.959194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:33.959194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:33.959194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:33.959194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:33.959194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.959194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:33.959194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:33.959194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.959194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:33.959610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.959610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:33.959850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:33.960329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:33.960873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:33.960873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:33.960873 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:36.214284 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:36.215317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:36.215317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:36.216843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:36.251270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:36.251630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:36.252489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:36.252489 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:36.252489 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:36.252489 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:36.252489 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:36.252489 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:36.252489 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:36.252489 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:36.253413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:36.253413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:36.253413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:36.253413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:36.253413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:36.253413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:36.253413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:36.253413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:36.253413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:36.253413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:36.253413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:36.253413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:36.254035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:36.254035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:36.254035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:36.254035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:36.254035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:36.254035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:36.254035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:36.254035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:36.254035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:36.254035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:36.254035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:36.254035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:36.254035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:36.254674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:36.255672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:36.257420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:36.257420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:36.257420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:36.257420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:36.257420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:36.257420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:38.912364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:38.913337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:38.913451 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:38.914666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:38.915032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:38.915337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.916426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:38.916426 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.916426 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:38.916426 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:38.916426 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.916426 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:38.916426 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.916426 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:38.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:38.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:38.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:38.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:38.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:38.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:38.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:38.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:38.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:38.918128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.919030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:38.919030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:38.919030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:38.919030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:38.919030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:38.919030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:38.919030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:38.919030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:38.919030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.919030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:38.919030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:38.919030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.919030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:38.919885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.919885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:38.920524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:38.921028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.921585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:38.921585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.921585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:38.946436 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:38.946814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:38.947080 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:38.947429 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:01:38.948023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:38.948350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.949443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:38.949443 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.949443 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:38.949443 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:38.949443 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.949443 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:38.949443 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.949443 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:38.951156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:38.951645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:38.951645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:38.951645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.951645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:38.951645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:38.951645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.951645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:38.951645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.951645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:38.952729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.952729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:38.952729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:38.952729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:38.952729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:38.952729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:38.952729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:38.952729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:01:38.952729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:38.952729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:38.952729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.952729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:38.952977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:38.953397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.953397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:38.953397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.953397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:38.953397 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:38.954282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:38.954589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:38.954918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.954918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:38.954918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:38.954918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.954918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:38.954918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.954918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:38.954918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:38.954918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:38.954918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:38.956513 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:38.976560 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:38.976776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.977484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:38.977484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.977484 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:38.977484 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:38.977484 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.977484 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:38.977484 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.977484 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:38.978731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:38.979086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:38.979086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:38.979086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.979086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:38.979086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:38.979086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.979086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:38.979086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.979086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:38.980132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.024 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.980132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:38.980132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:38.980132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:38.980132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:38.980132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:38.980132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:38.980132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:38.980132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:38.980132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.980132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:38.980493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:38.980844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.980844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:38.980844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.980844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:38.982735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:38.983086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:38.983439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.983439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:38.983439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.983439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:38.983439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:38.983439 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:38.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:38.984124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:38.984357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:38.984477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.985008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:38.985008 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.985008 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:38.985008 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:38.985008 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.985008 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:38.985008 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.985008 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:38.986388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.986388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:38.986388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:38.986388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:38.986388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:38.986388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:38.986388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:38.986388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.986388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:38.986388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:38.986388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:38.986388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.986735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:38.986735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:38.986735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:38.986735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:38.986735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:38.986735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:38.986735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:38.986735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:38.986735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.986735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:38.986735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:38.986735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.986735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:38.987058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.987058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:38.988024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:38.988656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:38.989172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:38.989172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:38.989172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:41.215026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:41.216082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:41.216082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:41.216368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:41.249729 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:41.250144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:41.251092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:41.251092 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:41.251092 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:41.251092 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:41.251092 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:41.251092 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:41.251092 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:41.251092 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:41.252141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:41.252141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:41.252141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:41.252141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:41.252141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:41.252141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:41.252141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:41.252141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:41.252141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:41.252141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:41.252141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:41.252141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:41.253180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:41.253180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:41.253180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:41.253180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:41.253180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:41.253180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:41.253180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:41.253180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:41.253180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:41.253180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:41.253180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:41.253180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:41.253180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:41.253861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:41.254655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:41.255643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:41.255643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:41.255643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:41.255643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:41.255643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:41.255643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:43.913020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:43.913686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:43.913686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:43.914288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:43.914634 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:43.914887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.915638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:43.915638 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.915638 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:43.915638 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:43.915638 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.915638 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:43.915638 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.915638 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:43.916437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.916437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:43.916437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:43.916437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:43.916437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:43.916437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:43.916437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:43.916437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.916437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:43.916437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:43.916437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:43.916437 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.916836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:43.916836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:43.916836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:43.916836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:43.916836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:43.916836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:43.916836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:43.916836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:43.916836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.916836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:43.916836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:43.916836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.916836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:43.917255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:43.917710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.918312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:43.918312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:43.918312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.918312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:43.918312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.918312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:43.946711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:43.947067 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:43.947192 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:43.947332 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:01:43.947868 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:43.948152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.948871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:43.948871 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.948871 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:43.948871 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:43.948871 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.948871 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:43.948871 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.948871 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:43.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:43.951067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:43.952136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:43.952136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:43.952136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:01:43.952136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:43.952136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:43.952136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.952136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:43.952136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:43.952136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:43.952136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.952136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:43.952136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:43.952136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:43.952901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:43.952901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:43.952901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:43.952901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:43.952901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:43.952901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.952901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:43.952901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:43.952901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.952901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:43.952901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.952901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:43.953332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:43.953791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:43.954315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.954315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:43.954315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:43.954315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.954315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:43.954315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.954315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:43.954699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:43.955058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:43.955217 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:43.955420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:43.955873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:43.956262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.956975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:43.956975 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.956975 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:43.956975 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:43.956975 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.956975 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:43.956975 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.956975 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:43.957829 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:43.958456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:43.958456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:43.958456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.958456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:43.958456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:43.958456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.958456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:43.958456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.958456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:43.958865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.958865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:43.958865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:43.958865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:43.958865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:43.958865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:43.958865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:43.958865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:43.958865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:43.958865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.958865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:43.959476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:43.959968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.959968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:43.959968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.959968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:43.960342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:43.960839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:43.961445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.961445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:43.961445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.961445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:43.983313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:43.983861 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:43.984027 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:43.984335 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:43.984848 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:43.985241 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:43.985472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.986225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:43.986225 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.986225 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:43.986225 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:43.986225 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.986225 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:43.986225 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.986225 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:43.987848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.987848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:43.987848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:43.987848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:43.987848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:43.987848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:43.987848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:43.987848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.987848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:43.987848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:43.987848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:43.987848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.988447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:43.988447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:43.988447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:43.988447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:43.988447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:43.988447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:43.988447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:43.988447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:43.988447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.988447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:43.988447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:43.988447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.988447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:43.989046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:43.989761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:43.990631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:43.990631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:43.990631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:43.990631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:43.990631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:43.990631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:46.215667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:46.216510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:46.216510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:46.217533 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:46.251555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:46.251807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:46.252342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:46.252342 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:46.252342 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:46.252342 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:46.252342 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:46.252342 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:46.252342 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:46.252342 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:46.252989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:46.252989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:46.252989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:46.252989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:46.252989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:46.252989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:46.252989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:46.252989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:46.252989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:46.252989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:46.252989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:46.252989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:46.253580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:46.253580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:46.253580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:46.253580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:46.253580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:46.253580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:46.253580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:46.253580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:46.253580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:46.253580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:46.253580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:46.253580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:46.253580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:46.254186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:46.254864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:46.255701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:46.255701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:46.255701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:46.255701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:46.255701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:46.255701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:48.913781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:48.914357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:48.914474 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:48.915511 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:48.915826 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:48.916123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.916958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:48.916958 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.916958 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:48.916958 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:48.916958 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.916958 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:48.916958 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.916958 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:48.918565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.918565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:48.918565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:48.918565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:48.918565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:48.918565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:48.918565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:48.918565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.918565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:48.918565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:48.918565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:48.918565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.918902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:48.918902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:48.918902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:48.918902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:48.918902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:48.918902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:48.918902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:48.918902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:48.918902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.918902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:48.918902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:48.918902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.918902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:48.919858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.919858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:48.920151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:48.920935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.921441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:48.921441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.921441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:48.947223 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:48.947650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:48.947848 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:48.948177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:01:48.948875 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:48.949359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.950136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:48.950136 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.950136 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:48.950136 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:48.950136 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.950136 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:48.950136 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.950136 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:48.951442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:48.951812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:48.951812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:48.951812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.951812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:48.951812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:48.951812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.951812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:48.951812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.951812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:48.952672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.952672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:48.952672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:48.952672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:48.952672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:48.952672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:48.952672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:48.952672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:01:48.952672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:48.952672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:48.952672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.952672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:48.953019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:48.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:48.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:48.954376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:48.954775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:48.955181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.955181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:48.955181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:48.955181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.955181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:48.955181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.955181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:48.955181 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:48.955406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:48.955522 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:48.955675 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:48.956167 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:48.956583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.957456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:48.957456 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.957456 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:48.957456 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:48.957456 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.957456 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:48.957456 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.957456 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:48.958544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:48.958943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:48.958943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:48.958943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.958943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:48.958943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:48.958943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.958943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:48.958943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.958943 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:48.959368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.959368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:48.959368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:48.959368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:48.959368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:48.959368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:48.959368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:48.959368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:48.959368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:48.959368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.959368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:48.959692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:48.960008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.960008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:48.960008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.960008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:48.960523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:48.960934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:48.961481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.961481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:48.961481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.961481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:48.983925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:48.984392 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:48.984510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:48.984823 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:48.985381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:48.985748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:48.986002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.986824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:48.986824 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.986824 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:48.986824 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:48.986824 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.986824 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:48.986824 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.986824 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:48.988614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.988614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:48.988614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:48.988614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:48.988614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:48.988614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:48.988614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:48.988614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.988614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:48.988614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:48.988614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:48.988614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.989000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:48.989000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:48.989000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:48.989000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:48.989000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:48.989000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:48.989000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:48.989000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:48.989000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.989000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:48.989000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:48.989000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.989000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:48.989832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:48.990274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:48.990987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:48.990987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:48.990987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:48.990987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:48.990987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:48.990987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:51.216298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:51.216968 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:51.216968 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:51.217950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:51.257764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:51.258531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:51.258918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:51.258918 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:51.258918 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:51.258918 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:51.258918 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:51.258918 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:51.258918 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:51.258918 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:51.259900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:51.259900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:51.259900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:51.259900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:51.259900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:51.259900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:51.259900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:51.259900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:51.259900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:51.259900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:51.259900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:51.259900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:51.260876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:51.260876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:51.260876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:51.260876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:51.260876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:51.260876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:51.260876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:51.260876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:51.260876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:51.260876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:51.260876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:51.260876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:51.260876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:51.261693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:51.262832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:51.263802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:51.263802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:51.263802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:51.263802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:51.263802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:51.263802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:53.914480 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:53.915076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:53.915373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:53.916541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:53.916879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:53.917124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.918014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:53.918014 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.918014 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:53.918014 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:53.918014 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.918014 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:53.918014 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.918014 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:53.919680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.919680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:53.919680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:53.919680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:53.919680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:53.919680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:53.919680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:53.919680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.919680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:53.919680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:53.919680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:53.919680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.920296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:53.920296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:53.920296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:53.920296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:53.920296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:53.920296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:53.920296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:53.920296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:53.920296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.920296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:53.920296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:53.920296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.920296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:53.920830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.920830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:53.921542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:53.922236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.923051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:53.923051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.923051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:53.947847 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:53.948487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:53.948713 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:53.948971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:01:53.949720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:53.950055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.951106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:53.951106 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.951106 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:53.951106 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:53.951106 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.951106 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:53.951106 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.951106 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:53.952592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:53.953371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:53.953371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:53.953371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.953371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:53.953371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:53.953371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.953371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:53.953371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.953371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:53.954026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.954026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:53.954026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:53.954026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:53.954026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:53.954026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:53.954026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:53.954026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:01:53.954026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:53.954026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:53.954026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.954026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:53.954644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:53.955265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.955265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:53.955265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.955265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:53.955869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:53.956631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:53.957491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.957491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:53.957491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:53.957491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.957491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:53.957491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.957491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:53.957491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:53.957491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:53.957491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:53.958029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:53.958029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:53.958029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.958836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:53.958836 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.958836 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:53.958836 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:53.958836 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.958836 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:53.958836 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.958836 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:53.959810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:53.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:53.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:53.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:53.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:53.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:53.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:53.960886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.960886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:53.960886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:53.960886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:53.960886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:53.960886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:53.960886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:53.960886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:53.960886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:53.960886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.960886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:53.961443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:53.962014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:53.962689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:53.963533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:53.963533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:53.963533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:53.963533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:53.963533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.963533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:53.963533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:53.963533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.963533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:53.963533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.963533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:53.984869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:53.985174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:53.985401 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:53.985797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:53.986333 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:53.986758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:53.986856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.987560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:53.987560 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.987560 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:53.987560 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:53.987560 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.987560 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:53.987560 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.987560 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:53.989037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.989037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:53.989037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:53.989037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:53.989037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:53.989037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:53.989037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:53.989037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.989037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:53.989037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:53.989037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:53.989037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.989489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:53.989489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:53.989489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:53.989489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:53.989489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:53.989489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:53.989489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:53.989489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:53.989489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.989489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:53.989489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:53.989489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.989489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:53.989851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.989851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:53.990664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:53.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:53.991898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:53.991898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:53.991898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:56.216994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:56.217993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:56.217993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:56.219356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:56.253319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:56.253780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:56.254644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:56.254644 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:56.254644 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:56.254644 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:56.254644 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:56.254644 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:56.254644 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:56.254644 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:56.255511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:56.255511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:56.255511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:56.255511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:56.255511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:56.255511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:56.255511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:56.255511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:56.255511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:56.255511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:56.255511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:56.255511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:56.256227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:56.256227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:56.256227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:56.256227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:56.256227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:56.256227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:56.256227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:56.256227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:56.256227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:56.256227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:56.256227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:56.256227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:56.256227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:56.256930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:56.257811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:56.258878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:56.258878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:56.258878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:56.258878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:56.258878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:56.258878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:58.915169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:58.916172 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:58.916172 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:58.917182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:58.917560 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:58.917811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.918729 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:58.920254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.920254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:58.920254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:58.920254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:58.920254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:58.920254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:58.920254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:58.920254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.920254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:58.920254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:58.920254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:58.920254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.921044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:58.921044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:58.921044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:58.921044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:58.921044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:58.921044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:58.921044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:58.921044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:58.921044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.921044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:58.921044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:58.921044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.921044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:58.921722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.921722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:58.921918 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:58.922963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.923916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:58.923916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.923916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:58.948364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:58.948733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:58.948915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:58.949100 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:01:58.949638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:58.949935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.951268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:58.951268 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.951268 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:58.951268 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:58.951268 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.951268 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:58.951268 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.951268 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:58.952473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:58.953576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:58.953576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:58.953576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.953576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:58.953576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:58.953576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.953576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:58.953576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.953576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:58.954238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.954238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:58.954238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:58.954238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:58.954238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:58.954238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:58.954238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:58.954238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:01:58.954238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:58.954238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:58.954238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.954238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:58.954906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:58.955734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.955734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:58.955734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.955734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:58.955913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:58.956988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:58.957816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.957816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:58.957816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:58.957816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.957816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:58.957816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.957816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:58.957816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:58.957816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:58.957816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:58.958352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:58.958352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:58.958693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.959607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:58.959607 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.959607 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:58.959607 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:58.959607 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.959607 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:58.959607 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.959607 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:01:58.960436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:01:58.961018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:58.961018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:01:58.961018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.961018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:01:58.961018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:01:58.961018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.961018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:01:58.961018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.961018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:01:58.961381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.961381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:58.961381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:58.961381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:58.961381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:58.961381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:58.961381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:58.961381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:58.961381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:58.961381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.961381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:58.961822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:58.962377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.962377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:58.962377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.962377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:58.962859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:58.963561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:58.964273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.964273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:58.964273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.964273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:01:58.985232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:01:58.985573 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:01:58.985704 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:01:58.985866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:01:58.986222 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:58.986419 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:01:58.986555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.987053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:01:58.987053 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.987053 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:01:58.987053 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:01:58.987053 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.987053 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:01:58.987053 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.987053 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:01:58.987909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.987909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:01:58.987909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:58.987909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:01:58.987909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:01:58.987909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:58.987909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:58.987909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.987909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:58.987909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:58.987909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:01:58.987909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:01:58.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:58.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:01:58.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:58.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:01:58.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:01:58.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:58.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:01:58.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:01:58.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:01:58.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:01:58.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:01:58.988908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:01:58.989388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:01:58.989889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:01:58.989889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:01:58.989889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:01.217612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:01.218697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:01.218697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:01.219529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:01.258354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:01.258621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:01.260366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:01.262961 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:01.263738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:01.265889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:01.267147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:01.267147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:01.267147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:01.267147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:01.267147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:01.267147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:01.267147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:01.267147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:01.267147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:03.915832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:03.917403 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:03.917403 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:03.917928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:03.917928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:03.918423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.919640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:03.919640 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.919640 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:03.919640 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:03.919640 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.919640 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:03.919640 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.919640 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:03.920906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.920906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:03.920906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:03.920906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:03.920906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:03.920906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:03.920906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:03.920906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.920906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:03.920906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:03.920906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:03.920906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.922088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:03.922088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:03.922088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:03.922088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:03.922088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:03.922088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:03.922088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:03.922088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:03.922088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.922088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:03.922088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:03.922088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.922088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:03.922799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:03.923446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.924201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:03.924201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:03.924201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.924201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:03.924201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.924201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:03.948993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:03.949444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:03.949573 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:03.949689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:02:03.950146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:03.950393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.951072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:03.951072 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.951072 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:03.951072 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:03.951072 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.951072 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:03.951072 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.951072 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:03.951959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:03.953586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:03.954242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:03.954242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:03.954242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:02:03.954242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:03.954242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:03.954242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.954242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:03.954242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:03.954242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:03.954242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.954242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:03.954242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:03.954242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:03.954792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:03.954792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:03.954792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:03.954792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:03.954792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:03.954792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.954792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:03.954792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:03.954792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.954792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:03.954792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.954792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:03.955358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:03.957439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:03.958688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.958688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:03.958688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:03.958688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.958688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:03.958688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.958688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:03.958688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:03.958688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:03.958688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:03.958992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:03.958992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:03.958992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.959814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:03.959814 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.959814 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:03.959814 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:03.959814 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.959814 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:03.959814 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.959814 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:03.960649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:03.961093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:03.961093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:03.961093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.961093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:03.961093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:03.961093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.961093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:03.961093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.961093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:03.961743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.961743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:03.961743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:03.961743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:03.961743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:03.961743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:03.961743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:03.961743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:03.961743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:03.961743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.961743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:03.962092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:03.962500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.962500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:03.962500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.962500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:03.963194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:03.963692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:03.964172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.964172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:03.964172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.964172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:03.985817 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:03.986188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:03.986332 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:03.986510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:03.986885 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:03.987230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:03.987479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.988097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:03.988097 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.988097 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:03.988097 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:03.988097 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.988097 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:03.988097 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.988097 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:03.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:03.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:03.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:03.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:03.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:03.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:03.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:03.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:03.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:03.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.989447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:03.989447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:03.989447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:03.989447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:03.989447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:03.989447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:03.989447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:03.989447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:03.989447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.989447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:03.989447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:03.989447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.989447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:03.989981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:03.990708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:03.991572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:03.991572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:03.991572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:03.991572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:03.991572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:03.991572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:06.218387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:06.219135 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:06.219135 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:06.220572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:06.259459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:06.259855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:06.260931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:06.260931 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:06.260931 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:06.260931 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:06.260931 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:06.260931 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:06.260931 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:06.260931 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:06.261614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:06.261614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:06.261614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:06.261614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:06.261614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:06.261614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:06.261614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:06.261614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:06.261614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:06.261614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:06.261614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:06.261614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:06.262005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:06.262005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:06.262005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:06.262005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:06.262005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:06.262005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:06.262005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:06.262005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:06.262005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:06.262005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:06.262005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:06.262005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:06.262005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:06.262377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:06.262777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:06.263280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:06.263280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:06.263280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:06.263280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:06.263280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:06.263280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:08.916526 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:08.917734 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:08.917734 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:08.918336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:08.918685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:08.918923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.919891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:08.919891 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.919891 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:08.919891 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:08.919891 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.919891 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:08.919891 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.919891 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:08.920744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.920744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:08.920744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:08.920744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:08.920744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:08.920744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:08.920744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:08.920744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.920744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:08.920744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:08.920744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:08.920744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.921416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:08.921416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:08.921416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:08.921416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:08.921416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:08.921416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:08.921416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:08.921416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:08.921416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.921416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:08.921416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:08.921416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.921416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:08.922112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:08.923109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.924114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:08.924114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:08.924114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.924114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:08.924114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.924114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:08.949659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:08.950130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:08.950417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:08.950656 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:02:08.951413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:08.951644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.952494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:08.952494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.952494 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:08.952494 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:08.952494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.952494 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:08.952494 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.952494 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:08.953824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:08.954646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:08.954646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:08.954646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.954646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:08.954646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:08.954646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.954646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:08.954646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.954646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:08.955337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.955337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:08.955337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:08.955337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:08.955337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:08.955337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:08.955337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:08.955337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:02:08.955337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:08.955337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:08.955337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.955337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:08.955937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:08.956536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.956536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:08.956536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.956536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:08.957075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:08.957804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:08.958631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.958631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:08.958631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:08.958631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.958631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:08.958631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.958631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:08.958631 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:08.958631 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:08.958631 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:08.959110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:08.959110 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:08.959110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.959736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:08.959736 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.959736 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:08.959736 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:08.959736 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.959736 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:08.959736 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.959736 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:08.960486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:08.960997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:08.960997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:08.960997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.960997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:08.960997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:08.960997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.960997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:08.960997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.960997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:08.961465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.961465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:08.961465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:08.961465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:08.961465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:08.961465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:08.961465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:08.961465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:08.961465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:08.961465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.961465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:08.961805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:08.962163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.962163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:08.962163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.962163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:08.962812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:08.963324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:08.963817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.963817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:08.963817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.963817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:08.986429 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:08.986839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:08.986995 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:08.987331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:08.987879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:08.988076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:08.988156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.988750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:08.988750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.988750 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:08.988750 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:08.988750 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.988750 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:08.988750 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.988750 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:08.990066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.990066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:08.990066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:08.990066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:08.990066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:08.990066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:08.990066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:08.990066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.990066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:08.990066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:08.990066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:08.990066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.990704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:08.990704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:08.990704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:08.990704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:08.990704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:08.990704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:08.990704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:08.990704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:08.990704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.990704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:08.990704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:08.990704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.990704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:08.991345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.991345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:08.991631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:08.992366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:08.993193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:08.993193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:08.993193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:11.219365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:11.220263 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:11.220263 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:11.221422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:11.234580 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=242 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 16:02:11.256147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:11.256571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:11.257625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:11.257625 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:11.257625 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:11.257625 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:11.257625 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:11.257625 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:11.257625 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:11.257625 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:11.258499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:11.258499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:11.258499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:11.258499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:11.258499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:11.258499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:11.258499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:11.258499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:11.258499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:11.258499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:11.258499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:11.258499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:11.258892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:11.258892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:11.258892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:11.258892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:11.258892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:11.258892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:11.258892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:11.258892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:11.258892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:11.258892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:11.258892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:11.258892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:11.258892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:11.259272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:11.259670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:11.260147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:11.260147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:11.260147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:11.260147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:11.260147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:11.260147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:13.917184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:13.917909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:13.918155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:13.919243 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:13.919601 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:13.919883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.920852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:13.920852 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.920852 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:13.920852 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:13.920852 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.920852 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:13.920852 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.920852 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:13.922405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.922405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:13.922405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:13.922405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:13.922405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:13.922405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:13.922405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:13.922405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.922405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:13.922405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:13.922405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:13.922405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.922758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:13.922758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:13.922758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:13.922758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:13.922758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:13.922758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:13.922758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:13.922758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:13.922758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.922758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:13.922758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:13.922758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.922758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:13.923186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.923186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:13.923999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:13.924516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.925073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:13.925073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.925073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:13.950275 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:13.950895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:13.951087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:13.951313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:02:13.951876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:13.952225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.953182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:13.953182 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.953182 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:13.953182 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:13.953182 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.953182 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:13.953182 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.953182 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:13.954685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:13.955344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:13.955344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:13.955344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.955344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:13.955344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:13.955344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.955344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:13.955344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.955344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:13.956317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.956317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:13.956317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:13.956317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:13.956317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:13.956317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:13.956317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:13.956317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:02:13.956317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:13.956317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:13.956317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.956317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:13.956722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:13.957160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.957160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:13.957160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.957160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:13.958022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:13.958534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:13.959062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.959062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:13.959062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:13.959062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.959062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:13.959062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.959062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:13.959062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:13.959062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:13.959062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:13.959421 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:13.959667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:13.960041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.960831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:13.960831 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.960831 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:13.960831 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:13.960831 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.960831 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:13.960831 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.960831 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:13.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:13.962307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:13.962307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:13.962307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.962307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:13.962307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:13.962307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.962307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:13.962307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.962307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:13.962753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.962753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:13.962753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:13.962753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:13.962753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:13.962753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:13.962753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:13.962753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:13.962753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:13.962753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.962753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:13.963112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:13.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:13.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:13.963807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:13.964378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:13.964923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.964923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:13.964923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.964923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:13.986782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:13.987335 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:13.987527 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:13.987804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:13.988327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:13.988694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:13.988925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.989749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:13.989749 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.989749 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:13.989749 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:13.989749 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.989749 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:13.989749 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.989749 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:13.991174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.991174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:13.991174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:13.991174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:13.991174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:13.991174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:13.991174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:13.991174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.991174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:13.991174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:13.991174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:13.991174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.991568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:13.991568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:13.991568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:13.991568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:13.991568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:13.991568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:13.991568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:13.991568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:13.991568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.991568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:13.991568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:13.991568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.991568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:13.991945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.991945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:13.992745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:13.993238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:13.995511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:13.995511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:13.995511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:16.220109 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:16.220860 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:16.220860 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:16.221763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:16.256793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:16.257097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:16.257855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:16.257855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:16.257855 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:16.257855 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:16.257855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:16.257855 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:16.257855 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:16.257855 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:16.258469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:16.258469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:16.258469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:16.258469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:16.258469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:16.258469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:16.258469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:16.258469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:16.258469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:16.258469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:16.258469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:16.258469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:16.258877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:16.258877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:16.258877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:16.258877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:16.258877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:16.258877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:16.258877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:16.258877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:16.258877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:16.258877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:16.258877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:16.258877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:16.258877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:16.259243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:16.259679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:16.260255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:16.260255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:16.260255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:16.260255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:16.260255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:16.260255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:18.918003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:18.918952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:18.918952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:18.919841 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:18.920246 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:18.920511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.921581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:18.921581 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.921581 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:18.921581 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:18.921581 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.921581 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:18.921581 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.921581 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:18.922881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.922881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:18.922881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:18.922881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:18.922881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:18.922881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:18.922881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:18.922881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.922881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:18.922881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:18.922881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:18.922881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.923405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:18.923405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:18.923405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:18.923405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:18.923405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:18.923405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:18.923405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:18.923405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:18.923405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.923405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:18.923405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:18.923405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.923405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:18.923817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.923817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:18.924583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:18.925116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.925704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:18.925704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.925704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:18.950740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:18.951107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:18.951310 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:18.951674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:02:18.952135 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:18.952453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.953227 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:18.953227 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.953227 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:18.953227 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:18.953227 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.953227 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:18.953227 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.953227 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:18.954403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:18.955161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:18.955161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:18.955161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.955161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:18.955161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:18.955161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.955161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:18.955161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.955161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:18.955901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.955901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:18.955901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:18.955901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:18.955901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:18.955901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:18.955901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:18.955901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:02:18.955901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:18.955901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:18.955901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.955901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:18.956540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:18.957143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.957143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:18.957143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.957143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:18.957353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:18.958080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:18.958887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.958887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:18.958887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:18.958887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.958887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:18.958887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.958887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:18.958887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:18.959523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:18.959523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:18.959523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:18.960026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:18.960442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.961159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:18.961159 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.961159 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:18.961159 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:18.961159 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.961159 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:18.961159 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.961159 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:18.961910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:18.962722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:18.963137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:18.963137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:18.963137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:18.963137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:18.963137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.963137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:18.963137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:18.963137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:18.963137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.963137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:18.963137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:18.963137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:18.963137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:18.963517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:18.963517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:18.963517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:18.963517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:18.963517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.963517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:18.963517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:18.963517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.963517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:18.963517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.963517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:18.963845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:18.964348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:18.964858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.964858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:18.964858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.964858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:18.987355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:18.987734 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:18.987916 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:18.988187 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:18.988758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:18.989101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:18.989356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.990145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:18.990145 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.990145 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:18.990145 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:18.990145 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.990145 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:18.990145 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.990145 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:18.991511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.991511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:18.991511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:18.991511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:18.991511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:18.991511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:18.991511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:18.991511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.991511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:18.991511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:18.991511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:18.991511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.992400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:18.992400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:18.992400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:18.992400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:18.992400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:18.992400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:18.992400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:18.992400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:18.992400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.992400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:18.992400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:18.992400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.992400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:18.993118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:18.993908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:18.995022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:18.995022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:18.995022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:18.995022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:18.995022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:18.995022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:21.220817 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:21.221894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:21.222121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:21.223461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:21.251279 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:21.251636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:21.252626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:21.252626 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:21.252626 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:21.252626 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:21.252626 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:21.252626 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:21.252626 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:21.252626 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:21.254362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:21.254362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:21.254362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:21.254362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:21.254362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:21.254362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:21.254362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:21.254362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:21.254362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:21.254362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:21.254362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:21.254362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:21.254897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:21.254897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:21.254897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:21.254897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:21.254897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:21.254897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:21.254897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:21.254897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:21.254897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:21.254897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:21.254897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:21.254897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:21.254897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:21.255456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:21.255456 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:21.256599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:21.257291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:21.258083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:21.258083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:21.258083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:23.918699 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:23.919590 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:23.919819 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:23.920931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:23.921366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:23.921756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.922505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:23.922505 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.922505 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:23.922505 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:23.922505 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.922505 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:23.922505 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.922505 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:23.924191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.924191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:23.924191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:23.924191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:23.924191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:23.924191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:23.924191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:23.924191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.924191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:23.924191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:23.924191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:23.924191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.924834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:23.924834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:23.924834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:23.924834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:23.924834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:23.924834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:23.924834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:23.924834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:23.924834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.924834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:23.924834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:23.924834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.924834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:23.925439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.925439 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:23.925945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:23.926728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.927609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:23.927609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.927609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:23.951159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:23.951576 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:23.951753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:23.952128 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:02:23.952481 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:23.952770 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.953657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:23.953657 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.953657 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:23.953657 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:23.953657 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.953657 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:23.953657 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.953657 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:23.954929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:23.955669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:23.955669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:23.955669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.955669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:23.955669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:23.955669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.955669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:23.955669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.955669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:23.956047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.956047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:23.956047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:23.956047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:23.956047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:23.956047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:23.956047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:23.956047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:02:23.956047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:23.956047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:23.956047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.956047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:23.956594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:23.957168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:23.957810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:23.958600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:23.958600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:23.958600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:23.958600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:23.958600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:23.958600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.958600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:23.958600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:23.958600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.958600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:23.958600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.958600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:23.959301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:23.959301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:23.959405 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:23.959536 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:23.959993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:23.960394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.961087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:23.961087 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.961087 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:23.961087 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:23.961087 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.961087 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:23.961087 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.961087 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:23.961949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:23.962667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:23.962667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:23.962667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.962667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:23.962667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:23.962667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.962667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:23.962667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.962667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:23.963041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.963041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:23.963041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:23.963041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:23.963041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:23.963041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:23.963041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:23.963041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:23.963041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:23.963041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.963041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:23.963548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:23.964140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:23.964780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:23.965593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:23.965593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:23.965593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:23.965593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:23.965593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.965593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:23.965593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:23.965593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.965593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:23.965593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.965593 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:23.987919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:23.988275 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:23.988375 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:23.988565 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:23.988925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:23.989222 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:23.989399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.989928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:23.989928 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.989928 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:23.989928 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:23.989928 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.989928 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:23.989928 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.989928 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:23.990959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.990959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:23.990959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:23.990959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:23.990959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:23.990959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:23.990959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:23.990959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.990959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:23.990959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:23.990959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:23.990959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.991636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:23.991636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:23.991636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:23.991636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:23.991636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:23.991636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:23.991636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:23.991636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:23.991636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.991636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:23.991636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:23.991636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.991636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:23.992269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:23.992903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:23.993653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:23.993653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:23.993653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:23.993653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:23.993653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:23.993653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:26.222144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:26.222971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:26.222971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:26.223948 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:26.257226 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:26.257472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:26.258116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:26.258116 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:26.258116 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:26.258116 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:26.258116 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:26.258116 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:26.258116 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:26.258116 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:26.259191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:26.259191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:26.259191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:26.259191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:26.259191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:26.259191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:26.259191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:26.259191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:26.259191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:26.259191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:26.259191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:26.259191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:26.259693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:26.259693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:26.259693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:26.259693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:26.259693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:26.259693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:26.259693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:26.259693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:26.259693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:26.259693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:26.259693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:26.259693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:26.259693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:26.260194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:26.260194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:26.260466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:26.260924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:26.261535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:26.261535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:26.261535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:28.919596 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:28.920459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:28.920459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:28.921403 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:28.921807 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:28.922118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.923090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:28.923090 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.923090 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:28.923090 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:28.923090 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.923090 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:28.923090 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.923090 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:28.924613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.924613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:28.924613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:28.924613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:28.924613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:28.924613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:28.924613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:28.924613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.924613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:28.924613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:28.924613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:28.924613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.925312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:28.925312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:28.925312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:28.925312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:28.925312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:28.925312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:28.925312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:28.925312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:28.925312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.925312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:28.925312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:28.925312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.925312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:28.925957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:28.926986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.928090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:28.928090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:28.928090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.928090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:28.928090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.928090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:28.951620 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:28.951948 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:28.952117 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:28.952325 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:02:28.952854 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:28.953519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:28.954149 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.954149 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:28.954149 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:28.954149 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.954149 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:28.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:28.955295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:28.956088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:28.956088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:28.956088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.956088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:28.956088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:28.956088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.956088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:28.956088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.956088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:28.956560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.956560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:28.956560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:28.956560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:28.956560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:28.956560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:28.956560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:28.956560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:02:28.956560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:28.956560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:28.956560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.956560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:28.957137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:28.957719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.957719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:28.957719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.957719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:28.957895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:28.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:28.959430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.959430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:28.959430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:28.959430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.959430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:28.959430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.959430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:28.959745 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:28.959832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:28.959934 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:28.960060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:28.960550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:28.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.961539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:28.961539 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.961539 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:28.961539 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:28.961539 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.961539 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:28.961539 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.961539 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:28.962228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:28.962914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:28.962914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:28.962914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.962914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:28.962914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:28.962914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.962914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:28.962914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.962914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:28.963337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.963337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:28.963337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:28.963337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:28.963337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:28.963337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:28.963337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:28.963337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:28.963337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:28.963337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.963337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:28.963891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:28.964540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:28.965237 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:28.966230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:28.966230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:28.966230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:28.966230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:28.966230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.966230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:28.966230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:28.966230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.966230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:28.966230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.966230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:28.988411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:28.988767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:28.988962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:28.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:28.989511 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:28.989729 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:28.989857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.990489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:28.990489 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.990489 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:28.990489 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:28.990489 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.990489 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:28.990489 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.990489 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:28.991869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.991869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:28.991869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:28.991869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:28.991869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:28.991869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:28.991869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:28.991869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.991869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:28.991869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:28.991869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:28.991869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.992657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:28.992657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:28.992657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:28.992657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:28.992657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:28.992657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:28.992657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:28.992657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:28.992657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.992657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:28.992657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:28.992657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.992657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:28.993336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.993336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:28.993592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:28.994363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:28.995189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:28.995189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:28.995189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:31.222837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:31.223672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:31.223672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:31.224988 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:31.254959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:31.255383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:31.256274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:31.256274 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:31.256274 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:31.256274 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:31.256274 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:31.256274 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:31.256274 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:31.256274 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:31.257562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:31.257562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:31.257562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:31.257562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:31.257562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:31.257562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:31.257562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:31.257562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:31.257562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:31.257562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:31.257562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:31.257562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:31.257937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:31.257937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:31.257937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:31.257937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:31.257937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:31.257937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:31.257937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:31.257937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:31.257937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:31.257937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:31.257937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:31.257937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:31.257937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:31.258306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:31.258306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:31.259736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:31.260273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:31.260780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:31.260780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:31.260780 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:33.920418 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:33.921200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:33.921329 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:33.922673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:33.922851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:33.923099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.924144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:33.924144 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.924144 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:33.924144 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:33.924144 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.924144 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:33.924144 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.924144 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:33.925654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.925654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:33.925654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:33.925654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:33.925654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:33.925654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:33.925654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:33.925654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.925654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:33.925654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:33.925654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:33.925654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.926351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:33.926351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:33.926351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:33.926351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:33.926351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:33.926351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:33.926351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:33.926351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:33.926351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.926351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:33.926351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:33.926351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.926351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:33.926913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.926913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:33.927595 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:33.928385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.929886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:33.929886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.929886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:33.952459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:33.952794 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:33.952895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:33.952960 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:02:33.954424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:33.954424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.954618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:33.954618 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.954618 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:33.954618 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:33.954618 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.954618 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:33.954618 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.954618 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:33.955310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:33.956453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:33.957040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:33.957040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:33.957040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:02:33.957040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:33.957040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:33.957040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.957040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:33.957040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:33.957040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:33.957040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.957040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:33.957040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:33.957040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:33.957577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:33.957577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:33.957577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:33.957577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:33.957577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:33.957577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.957577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:33.957577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:33.957577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.957577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:33.957577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.957577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:33.958044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:33.958970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:33.959776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.959776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:33.959776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:33.959776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.959776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:33.959776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.959776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:33.960072 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:33.960442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:33.960534 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:33.961020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:33.961107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:33.961454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.961893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:33.961893 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.961893 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:33.961893 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:33.961893 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.961893 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:33.961893 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.961893 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:33.962432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:33.963076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:33.963643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:33.963643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:33.963643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:33.963643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:33.963643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.963643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:33.963643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:33.963643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:33.963643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.963643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:33.963643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:33.963643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:33.963643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:33.963884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:33.963884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:33.963884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:33.963884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:33.963884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.963884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:33.963884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:33.963884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.963884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:33.963884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.963884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:33.963884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:33.964110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:33.989292 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:33.989520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:33.989520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:33.989643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:33.990071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:33.990478 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:33.990705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.991413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:33.991413 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.991413 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:33.991413 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:33.991413 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.991413 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:33.991413 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.991413 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:33.991999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.991999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:33.991999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:33.991999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:33.991999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:33.991999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:33.991999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:33.991999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.991999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:33.991999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:33.991999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:33.991999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.992425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:33.992425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:33.992425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:33.992425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:33.992425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:33.992425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:33.992425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:33.992425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:33.992425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.992425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:33.992425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:33.992425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.992425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:33.992778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:33.993284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:33.994089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:33.994089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:33.994089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:33.994089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:33.994089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:33.994089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:36.223579 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:36.224415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:36.224415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:36.225450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:36.259125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:36.259584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:36.260702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:36.260702 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:36.260702 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:36.260702 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:36.260702 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:36.260702 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:36.260702 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:36.260702 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:36.261534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:36.261534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:36.261534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:36.261534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:36.261534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:36.261534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:36.261534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:36.261534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:36.261534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:36.261534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:36.261534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:36.261534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:36.262140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:36.262140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:36.262140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:36.262140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:36.262140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:36.262140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:36.262140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:36.262140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:36.262140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:36.262140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:36.262140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:36.262140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:36.262140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:36.262584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:36.262584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:36.262822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:36.263481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:36.264025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:36.264025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:36.264025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:38.921134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:38.921928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:38.921928 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:38.922497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:38.922756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:38.922966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.923895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:38.923895 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.923895 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:38.923895 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:38.923895 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.923895 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:38.923895 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.923895 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:38.924738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.924738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:38.924738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:38.924738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:38.924738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:38.924738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:38.924738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:38.924738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.924738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:38.924738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:38.924738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:38.924738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.925181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:38.925181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:38.925181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:38.925181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:38.925181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:38.925181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:38.925181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:38.925181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:38.925181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.925181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:38.925181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:38.925181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.925181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:38.925710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.925710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:38.925815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:38.926356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.927098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:38.927098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.927098 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:38.952956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:38.953416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:38.953652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:38.953861 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:02:38.954483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:38.954805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.955677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:38.955677 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.955677 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:38.955677 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:38.955677 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.955677 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:38.955677 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.955677 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:38.957077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:38.957603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:38.957603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:38.957603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.957603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:38.957603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:38.957603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.957603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:38.957603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.957603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:38.958686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.958686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:38.958686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:38.958686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:38.958686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:38.958686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:38.958686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:38.958686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:02:38.958686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:38.958686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:38.958686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.958686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:38.959059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:38.959434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.959434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:38.959434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.959434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:38.959786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:38.960262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:38.960783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.960783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:38.960783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:38.960783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.960783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:38.960783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.960783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:38.960783 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:38.960783 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:38.961183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:38.961183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:38.961476 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:38.961941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.962748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:38.962748 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.962748 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:38.962748 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:38.962748 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.962748 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:38.962748 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.962748 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:38.963562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:38.964193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:38.964193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:38.964193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.964193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:38.964193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:38.964193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.964193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:38.964193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.964193 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:38.964585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.964585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:38.964585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:38.964585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:38.964585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:38.964585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:38.964585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:38.964585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:38.964585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:38.964585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.964585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:38.965107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:38.965759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:38.966438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:38.967075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:38.967075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:38.967075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:38.967075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:38.967075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.967075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:38.967075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:38.967075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.967075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:38.967075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.967075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:38.989837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:38.990170 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:38.990423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:38.990692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:38.991390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:38.991869 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:38.992231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:38.993070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.993070 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:38.993070 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:38.993070 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.993070 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:38.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:38.994686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.994686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:38.994686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:38.994686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:38.994686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:38.994686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:38.994686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:38.994686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.994686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:38.994686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:38.994686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:38.994686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.995443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:38.995443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:38.995443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:38.995443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:38.995443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:38.995443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:38.995443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:38.995443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:38.995443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.995443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:38.995443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:38.995443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.995443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:38.996047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.996047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:38.996752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:38.997678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:38.998715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:38.998715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:38.998715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:41.224144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:41.225723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:41.225825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:41.226681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:41.253097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:41.253326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:41.253856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:41.253856 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:41.253856 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:41.253856 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:41.253856 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:41.253856 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:41.253856 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:41.253856 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:41.254603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:41.254603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:41.254603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:41.254603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:41.254603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:41.254603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:41.254603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:41.254603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:41.254603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:41.254603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:41.254603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:41.254603 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:41.254989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:41.254989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:41.254989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:41.254989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:41.254989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:41.254989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:41.254989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:41.254989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:41.254989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:41.254989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:41.254989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:41.254989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:41.254989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:41.255410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:41.255861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:41.256396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:41.256396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:41.256396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:41.256396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:41.256396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:41.256396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:43.921724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:43.922240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:43.922276 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:43.923500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:43.923818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:43.924083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.924950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:43.924950 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.924950 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:43.924950 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:43.924950 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.924950 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:43.924950 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.924950 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:43.926458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.926458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:43.926458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:43.926458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:43.926458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:43.926458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:43.926458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:43.926458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.926458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:43.926458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:43.926458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:43.926458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.926707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:43.926707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:43.926707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:43.926707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:43.926707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:43.926707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:43.926707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:43.926707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:43.926707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.926707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:43.926707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:43.926707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.926707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:43.926955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.926955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:43.928137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:43.928468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.928811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:43.928811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.928811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:43.953639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:43.954051 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:43.954263 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:43.954451 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:02:43.955227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:43.955542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.956393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:43.956393 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.956393 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:43.956393 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:43.956393 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.956393 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:43.956393 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.956393 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:43.957576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:43.957875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:43.957875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:43.957875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.957875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:43.957875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:43.957875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.957875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:43.957875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.957875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:43.958904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.958904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:43.958904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:43.958904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:43.958904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:43.958904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:43.958904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:43.958904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:02:43.958904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:43.958904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:43.958904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.958904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:43.959144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:43.959417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.959417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:43.959417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.959417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:43.960667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:43.961114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:43.961505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.961505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:43.961505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:43.961505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.961505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:43.961505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.961505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:43.961505 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:43.961505 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:43.961678 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:43.961678 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:43.961957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:43.962187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.962680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:43.962680 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.962680 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:43.962680 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:43.962680 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.962680 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:43.962680 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.962680 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:43.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:43.963758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:43.963758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:43.963758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.963758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:43.963758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:43.963758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.963758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:43.963758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.963758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:43.964430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.964430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:43.964430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:43.964430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:43.964430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:43.964430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:43.964430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:43.964430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:43.964430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:43.964430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.964430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:43.964668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:43.965030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.965030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:43.965030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.965030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:43.965676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:43.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:43.966381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.966381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:43.966381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.966381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:43.990213 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:43.990661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:43.990837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:43.991553 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:43.991956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:43.992174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.992909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:43.992909 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.992909 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:43.992909 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:43.992909 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.992909 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:43.992909 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.992909 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:43.994434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.994434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:43.994434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:43.994434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:43.994434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:43.994434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:43.994434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:43.994434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.994434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:43.994434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:43.994434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:43.994434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:43.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:43.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:43.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:43.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:43.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:43.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:43.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:43.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:43.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:43.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:43.994970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.994970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:43.996001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:43.996349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:43.996825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:43.996825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:43.996825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:46.226035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:46.226816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:46.226816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:46.228122 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:46.262939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:46.263478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:46.264230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:46.264230 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:46.264230 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:46.264230 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:46.264230 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:46.264230 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:46.264230 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:46.264230 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:46.264903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:46.264903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:46.264903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:46.264903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:46.264903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:46.264903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:46.264903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:46.264903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:46.264903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:46.264903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:46.264903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:46.264903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:46.265510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:46.265510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:46.265510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:46.265510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:46.265510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:46.265510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:46.265510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:46.265510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:46.265510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:46.265510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:46.265510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:46.265510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:46.265510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:46.266037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:46.266710 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:46.267476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:46.267476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:46.267476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:46.267476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:46.267476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:46.267476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:48.922346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:48.922942 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:48.922942 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:48.924189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:48.924541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:48.924802 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.925567 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:48.925567 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.925567 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:48.925567 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:48.925567 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.925567 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:48.925567 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.925567 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:48.926504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.926504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:48.926504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:48.926504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:48.926504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:48.926504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:48.926504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:48.926504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.926504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:48.926504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:48.926504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:48.926504 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.927049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:48.927049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:48.927049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:48.927049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:48.927049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:48.927049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:48.927049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:48.927049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:48.927049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.927049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:48.927049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:48.927049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.927049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:48.927639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:48.928382 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.929440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:48.929440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:48.929440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.929440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:48.929440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.929440 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:48.954135 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:48.954545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:48.954732 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:48.954913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:02:48.955508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:48.955794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.956494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:48.956494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.956494 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:48.956494 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:48.956494 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.956494 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:48.956494 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.956494 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:48.957625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:48.958325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:48.958325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:48.958325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.958325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:48.958325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:48.958325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.958325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:48.958325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.958325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:48.958977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.958977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:48.958977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:48.958977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:48.958977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:48.958977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:48.958977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:48.958977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:02:48.958977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:48.958977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:48.958977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.958977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:48.959539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:48.960119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:48.960879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:48.961756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:48.962631 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:48.962631 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:48.962631 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:48.962793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.963558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:48.963558 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.963558 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:48.963558 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:48.963558 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.963558 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:48.963558 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.963558 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:48.964278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:48.964716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:48.964716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:48.964716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.964716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:48.964716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:48.964716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.964716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:48.964716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.964716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:48.965106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.965106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:48.965106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:48.965106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:48.965106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:48.965106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:48.965106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:48.965106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:48.965106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:48.965106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.965106 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:48.965495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:48.965978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.965978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:48.965978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.965978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:48.966117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:48.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:48.967122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.967122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:48.967122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.967122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:48.990712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:48.991200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:48.991390 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:48.991612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:48.992069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:48.992426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:48.992640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.993395 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:48.993395 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.993395 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:48.993395 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:48.993395 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.993395 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:48.993395 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.993395 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:48.994752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.994752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:48.994752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:48.994752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:48.994752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:48.994752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:48.994752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:48.994752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.994752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:48.994752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:48.994752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:48.994752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.995472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:48.995472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:48.995472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:48.995472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:48.995472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:48.995472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:48.995472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:48.995472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:48.995472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.995472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:48.995472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:48.995472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.995472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:48.996194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:48.996866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:48.997727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:48.997727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:48.997727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:48.997727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:48.997727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:48.997727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:51.226811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:51.227367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:51.227367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:51.228118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:51.260491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:51.260795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:51.261697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:51.261697 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:51.261697 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:51.261697 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:51.261697 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:51.261697 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:51.261697 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:51.261697 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:51.262564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:51.262564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:51.262564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:51.262564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:51.262564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:51.262564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:51.262564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:51.262564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:51.262564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:51.262564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:51.262564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:51.262564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:51.263119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:51.263119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:51.263119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:51.263119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:51.263119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:51.263119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:51.263119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:51.263119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:51.263119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:51.263119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:51.263119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:51.263119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:51.263119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:51.263669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:51.264300 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:51.265031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:51.265031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:51.265031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:51.265031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:51.265031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:51.265031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:53.922977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:53.923938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:53.923938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:53.924758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:53.925083 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:53.925325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:53.926360 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.926360 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:53.926360 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:53.926360 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.926360 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:53.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.926360 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:53.927728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.927728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:53.927728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:53.927728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:53.927728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:53.927728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:53.927728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:53.927728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.927728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:53.927728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:53.927728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:53.927728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.928169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:53.928169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:53.928169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:53.928169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:53.928169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:53.928169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:53.928169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:53.928169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:53.928169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.928169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:53.928169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:53.928169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.928169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:53.928597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.928597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:53.929534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:53.930045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.930635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:53.930635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.930635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:53.954608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:53.955011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:53.955233 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:53.955441 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:02:53.955880 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:53.956168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.957007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:53.957007 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.957007 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:53.957007 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:53.957007 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.957007 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:53.957007 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.957007 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:53.958277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:53.958859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:53.958859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:53.958859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.958859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:53.958859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:53.958859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.958859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:53.958859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.958859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:53.959661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.959661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:53.959661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:53.959661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:53.959661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:53.959661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:53.959661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:53.959661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:02:53.959661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:53.959661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:53.959661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.959661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:53.960042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:53.960418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.960418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:53.960418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.960418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:53.961340 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:53.961797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:53.962305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.962305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:53.962305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:53.962305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.962305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:53.962305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.962305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:53.962305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:53.962305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:53.962305 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:53.962597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:53.962807 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:53.963126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.963789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:53.963789 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.963789 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:53.963789 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:53.963789 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.963789 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:53.963789 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.963789 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:53.964508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:53.964941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:53.964941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:53.964941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.964941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:53.964941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:53.964941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.964941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:53.964941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.964941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:53.965376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.965376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:53.965376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:53.965376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:53.965376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:53.965376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:53.965376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:53.965376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:53.965376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:53.965376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.965376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:53.965733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:53.966094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.966094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:53.966094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.966094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:53.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:53.967029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:53.967821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.967821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:53.967821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.967821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:53.991443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:53.991728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:53.991846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:53.991993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:53.992391 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:53.992740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:53.992925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.993598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:53.993598 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.993598 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:53.993598 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:53.993598 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.993598 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:53.993598 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.993598 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:53.994535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.994535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:53.994535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:53.994535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:53.994535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:53.994535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:53.994535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:53.994535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.994535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:53.994535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:53.994535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:53.994535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.994920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:53.994920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:53.994920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:53.994920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:53.994920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:53.994920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:53.994920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:53.994920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:53.994920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.994920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:53.994920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:53.994920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.994920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:53.995267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.995267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:53.995388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:53.995848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:53.996371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:53.996371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:53.996371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:56.227422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:56.228316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:56.228316 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:56.229472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:56.264655 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:56.264975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:56.266083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:56.266083 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:56.266083 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:56.266083 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:56.266083 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:56.266083 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:56.266083 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:56.266083 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:56.266475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:56.266475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:56.266475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:56.266475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:56.266475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:56.266475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:56.266475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:56.266475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:56.266475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:56.266475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:56.266475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:56.266475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:56.266874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:56.266874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:56.266874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:56.266874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:56.266874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:56.266874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:56.266874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:56.266874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:56.266874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:56.266874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:56.266874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:56.266874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:56.266874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:56.267271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:56.267799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:56.268358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:56.268358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:56.268358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:56.268358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:56.268358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:56.268358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:58.923872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:58.924814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:58.925074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:58.927367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:58.927367 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:58.927527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.928697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:58.928697 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.928697 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:58.928697 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:58.928697 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.928697 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:58.928697 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.928697 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:58.931548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.931548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:58.931548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:58.931548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:58.931548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:58.931548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:58.931548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:58.931548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.931548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:58.931548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:58.931548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:58.931548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.932158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:58.932158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:58.932158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:58.932158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:58.932158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:58.932158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:58.932158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:58.932158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:58.932158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.932158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:58.932158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:58.932158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.932158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:58.932746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.932746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:58.933301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:58.934862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.936462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:58.936462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.936462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:58.955114 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:58.955674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:58.955812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:58.955929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:02:58.956682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:58.956915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.957612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:58.957612 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.957612 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:58.957612 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:58.957612 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.957612 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:58.957612 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.957612 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:58.958604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:58.959362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:58.959362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:58.959362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.959362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:58.959362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:58.959362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.959362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:58.959362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.959362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:58.959769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.959769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:58.959769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:58.959769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:58.959769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:58.959769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:58.959769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:58.959769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:02:58.959769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:58.959769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:58.959769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.959769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:58.960429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:58.961095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:58.961814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:58.962700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:58.963448 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:58.963448 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:58.963559 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:58.963690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.964265 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:58.964265 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.964265 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:58.964265 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:58.964265 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.964265 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:58.964265 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.964265 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:02:58.965608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:02:58.966405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:58.966405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:02:58.966405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.966405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:02:58.966405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:02:58.966405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.966405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:02:58.966405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.966405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:02:58.967053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.967053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:58.967053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:58.967053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:58.967053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:58.967053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:58.967053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:58.967053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:58.967053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:58.967053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.967053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:58.967470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:58.967822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.967822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:58.967822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.967822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:58.968878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:58.969402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:58.969900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.969900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:58.969900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.969900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:02:58.992075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:02:58.992452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:02:58.992565 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:02:58.992743 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:02:58.993121 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:58.993393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:02:58.993524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.994064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:02:58.994064 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.994064 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:02:58.994064 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:02:58.994064 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.994064 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:02:58.994064 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.994064 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:02:58.995089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.995089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:02:58.995089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:58.995089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:02:58.995089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:02:58.995089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:58.995089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:58.995089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.995089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:58.995089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:58.995089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:02:58.995089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.995828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:02:58.995828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:58.995828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:02:58.995828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:58.995828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:02:58.995828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:02:58.995828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:58.995828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:02:58.995828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.995828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:02:58.995828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:02:58.995828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.995828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:02:58.996574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:02:58.997391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:02:58.998659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:02:58.998659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:02:58.998659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:02:58.998659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:02:58.998659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:02:58.998659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:01.228080 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:01.228806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:01.228806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:01.229712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:01.262097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:01.262287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:01.262748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:01.262748 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:01.262748 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:01.262748 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:01.262748 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:01.262748 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:01.262748 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:01.262748 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:01.263452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:01.263452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:01.263452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:01.263452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:01.263452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:01.263452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:01.263452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:01.263452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:01.263452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:01.263452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:01.263452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:01.263452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:01.263986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:01.263986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:01.263986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:01.263986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:01.263986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:01.263986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:01.263986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:01.263986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:01.263986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:01.263986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:01.263986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:01.263986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:01.263986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:01.264548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:01.265154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:01.265984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:01.265984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:01.265984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:01.265984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:01.265984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:01.265984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:03.924690 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:03.925832 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:03.926018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:03.926971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:03.927321 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:03.927707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.928753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:03.928753 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.928753 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:03.928753 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:03.928753 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:03.928753 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:03.928753 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.928753 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:03.930316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.930316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:03.930316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:03.930316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:03.930316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:03.930316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:03.930316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:03.930316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.930316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:03.930316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:03.930316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:03.930316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.931057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:03.931057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:03.931057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:03.931057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:03.931057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:03.931057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:03.931057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:03.931057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:03.931057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.931057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:03.931057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:03.931057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:03.931057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:03.931693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.931693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:03.931981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:03.933592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:03.935337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:03.935337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.935337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:03.955647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:03.956057 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:03.956170 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:03.956336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:03:03.956793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:03.957073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.957781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:03.957781 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.957781 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:03.957781 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:03.957781 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:03.957781 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:03.957781 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.957781 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:03.958874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:03.959609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:03.960255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:03.960255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:03.960255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:03:03.960255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:03.960255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:03.960255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.960255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:03.960255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:03.960255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:03.960255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.960255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:03.960255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:03.960255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:03.960916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:03.960916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:03.960916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:03.960916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:03.960916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:03.960916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.960916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:03.960916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:03.960916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:03.960916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:03.960916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.960916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:03.961573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:03.962632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:03.963446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.963446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:03.963446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:03.963446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:03.963446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:03.963446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.963446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:03.963446 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:03.963446 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:03.963446 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:03.963886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:03.963886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:03.964301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.965006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:03.965006 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.965006 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:03.965006 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:03.965006 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:03.965006 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:03.965006 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.965006 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:03.965797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:03.966260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:03.966260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:03.966260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.966260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:03.966260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:03.966260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:03.966260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:03.966260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.966260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:03.966748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.966748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:03.966748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:03.966748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:03.966748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:03.966748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:03.966748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:03.966748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:03.966748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:03.966748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.966748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:03.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:03.967464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:03.967464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:03.967464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.967464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:03.967857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:03.968692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:03.969194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:03.969194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:03.969194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.969194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:03.992600 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:03.993175 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:03.993383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:03.993645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:03.994197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:03.994660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:03.994921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.995901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:03.995901 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.995901 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:03.995901 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:03.995901 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:03.995901 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:03.995901 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.995901 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:03.997331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.997331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:03.997331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:03.997331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:03.997331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:03.997331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:03.997331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:03.997331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.997331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:03.997331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:03.997331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:03.997331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.997947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:03.997947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:03.997947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:03.997947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:03.997947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:03.997947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:03.997947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:03.997947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:03.997947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.997947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:03.997947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:03.997947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:03.997947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:03.998585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.998585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:03.999041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:03.999826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:04.000708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:04.000708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:04.000708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:06.228879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:06.229799 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:06.229877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:06.231161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:06.274554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:06.275102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:06.276282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:06.276282 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:06.276282 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:06.276282 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:06.276282 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:06.276282 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:06.276282 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:06.276282 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:06.277458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:06.277458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:06.277458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:06.277458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:06.277458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:06.277458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:06.277458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:06.277458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:06.277458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:06.277458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:06.277458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:06.277458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:06.278092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:06.278092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:06.278092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:06.278092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:06.278092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:06.278092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:06.278092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:06.278092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:06.278092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:06.278092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:06.278092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:06.278092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:06.278092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:06.278766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:06.279508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:06.280389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:06.280389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:06.280389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:06.280389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:06.280389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:06.280389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:08.925906 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:08.926784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:08.926784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:08.927641 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:08.928079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:08.928199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.929091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:08.929091 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.929091 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:08.929091 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:08.929091 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:08.929091 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:08.929091 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.929091 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:08.930816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.930816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:08.930816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:08.930816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:08.930816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:08.930816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:08.930816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:08.930816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.930816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:08.930816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:08.930816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:08.930816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.931414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:08.931414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:08.931414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:08.931414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:08.931414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:08.931414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:08.931414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:08.931414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:08.931414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.931414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:08.931414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:08.931414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:08.931414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:08.931949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.931949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:08.932608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:08.933368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:08.934179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:08.934179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.934179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:08.956315 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:08.956912 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:08.957092 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:08.957373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:03:08.957944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:08.958269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.959083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:08.959083 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.959083 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:08.959083 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:08.959083 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:08.959083 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:08.959083 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.959083 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:08.962336 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:08.963015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:03:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:08.963878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:08.963878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:08.963878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:08.963878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:08.963878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:08.963878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.963878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:08.963878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:08.963878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:08.963878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:08.963878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.963878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:08.964910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:08.965429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:08.965985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.965985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:08.965985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:08.965985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:08.965985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:08.965985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.965985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:08.966231 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:08.966525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:08.966691 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:08.966818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:08.967396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:08.967661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.968220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:08.968220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.968220 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:08.968220 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:08.968220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:08.968220 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:08.968220 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.968220 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:08.969119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:08.969544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:08.969544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:08.969544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.969544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:08.969544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:08.969544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:08.969544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:08.969544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.969544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:08.970044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.970044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:08.970044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:08.970044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:08.970044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:08.970044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:08.970044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:08.970044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:08.970044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:08.970044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.970044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:08.970389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:08.970732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:08.970732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:08.970732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.970732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:08.971254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:08.971715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:08.972186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:08.972186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:08.972186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.972186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:08.993360 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:08.993845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:08.994041 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:08.994342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:08.994752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:08.995011 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:08.995159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.995795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:08.995795 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.995795 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:08.995795 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:08.995795 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:08.995795 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:08.995795 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.995795 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:08.997376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.997376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:08.997376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:08.997376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:08.997376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:08.997376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:08.997376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:08.997376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.997376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:08.997376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:08.997376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:08.997376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.997939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:08.997939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:08.997939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:08.997939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:08.997939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:08.997939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:08.997939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:08.997939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:08.997939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.997939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:08.997939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:08.997939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:08.997939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:08.998509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.998509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:08.998941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:08.999644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:09.000657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:09.000657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:09.000657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:11.229614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:11.230472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:11.230472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:11.231328 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:11.245340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 16:03:11.274724 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:11.275110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:11.275993 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:11.275993 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:11.275993 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:11.275993 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:11.275993 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:11.275993 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:11.275993 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:11.275993 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:11.276862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:11.276862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:11.276862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:11.276862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:11.276862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:11.276862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:11.276862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:11.276862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:11.276862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:11.276862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:11.276862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:11.276862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:11.277541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:11.277541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:11.277541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:11.277541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:11.277541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:11.277541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:11.277541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:11.277541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:11.277541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:11.277541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:11.277541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:11.277541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:11.277541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:11.278191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:11.278999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:11.279937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:11.279937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:11.279937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:11.279937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:11.279937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:11.279937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:13.926604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:13.927420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:13.927420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:13.928266 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:13.928628 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:13.928869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.930140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:13.930140 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:13.930140 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:13.930140 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:13.930140 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:13.930140 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:13.930140 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.930140 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:13.932044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.932044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:13.932044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:13.932044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:13.932044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:13.932044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:13.932044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:13.932044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.932044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:13.932044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:13.932044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:13.932044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.932695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:13.932695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:13.932695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:13.932695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:13.932695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:13.932695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:13.932695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:13.932695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:13.932695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:13.932695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:13.932695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:13.932695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:13.932695 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:13.933493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.933493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:13.933936 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:13.934871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:13.935669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:13.935669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.935669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:13.956785 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:13.957200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:13.957410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:13.957518 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:03:13.958152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:13.958489 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.959278 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:13.959278 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:13.959278 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:13.959278 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:13.959278 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:13.959278 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:13.959278 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.959278 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:13.960236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:13.961329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:13.962338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:13.962338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:13.962338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:03:13.962338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:13.962338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:13.962338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.962338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:13.962338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:13.962338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:13.962338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.962338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:13.962338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:13.962338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:13.962994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:13.962994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:13.962994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:13.962994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:13.962994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:13.962994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:13.962994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:13.962994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:13.962994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:13.962994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:13.962994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.962994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:13.963481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:13.964118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:13.964874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:13.964874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:13.964874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:13.964874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:13.964874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:13.964874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.964874 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:13.966604 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:13.966933 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:13.967081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:13.967274 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:13.967723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:13.968078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.968845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:13.968845 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:13.968845 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:13.968845 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:13.968845 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:13.968845 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:13.968845 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.968845 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:13.969636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:13.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:13.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:13.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:13.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:13.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:13.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:13.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:13.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:13.970657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.970657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:13.970657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:13.970657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:13.970657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:13.970657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:13.970657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:13.970657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:13.970657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:13.970657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.970657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:13.971160 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:13.971743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:13.971743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:13.971743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.971743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:13.971921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:13.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:13.973475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:13.973475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:13.973475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.973475 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:13.993805 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:13.994668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:13.994778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:13.995060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:13.995819 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:13.996114 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:13.996297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.996999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:13.996999 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:13.996999 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:13.996999 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:13.996999 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:13.996999 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:13.996999 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.996999 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:13.998252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.998252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:13.998252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:13.998252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:13.998252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:13.998252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:13.998252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:13.998252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.998252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:13.998252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:13.998252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:13.998252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:13.999061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:13.999061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:13.999061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:13.999061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:13.999061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:13.999061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:13.999061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:13.999061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:13.999061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:13.999061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:13.999061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:13.999061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:13.999061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:13.999860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:14.000862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:14.002044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:14.002044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:14.002044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:14.002044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:14.002044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:14.002044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:16.230197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:16.230993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:16.230993 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:16.232180 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:16.264861 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:16.265383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:16.266248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:16.266248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:16.266248 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:16.266248 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:16.266248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:16.266248 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:16.266248 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:16.266248 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:16.267242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:16.267242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:16.267242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:16.267242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:16.267242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:16.267242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:16.267242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:16.267242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:16.267242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:16.267242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:16.267242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:16.267242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:16.267803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:16.267803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:16.267803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:16.267803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:16.267803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:16.267803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:16.267803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:16.267803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:16.267803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:16.267803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:16.267803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:16.267803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:16.267803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:16.268409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:16.269130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:16.270180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:16.270180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:16.270180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:16.270180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:16.270180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:16.270180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:18.927344 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:18.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:18.928089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:18.929151 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:18.929585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:18.929705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.930545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:18.930545 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:18.930545 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:18.930545 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:18.930545 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:18.930545 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:18.930545 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.930545 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:18.932020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.932020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:18.932020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:18.932020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:18.932020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:18.932020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:18.932020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:18.932020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.932020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:18.932020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:18.932020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:18.932020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.932564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:18.932564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:18.932564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:18.932564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:18.932564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:18.932564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:18.932564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:18.932564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:18.932564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:18.932564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:18.932564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:18.932564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:18.932564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:18.933147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.933147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:18.933711 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:18.934448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:18.935242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:18.935242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.935242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:18.957380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:18.957654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:18.957753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:18.957883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:03:18.958348 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:18.958625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.959273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:18.959273 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:18.959273 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:18.959273 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:18.959273 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:18.959273 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:18.959273 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.959273 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:18.960101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:18.960796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:18.960796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:18.960796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:18.960796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:18.960796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:18.960796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:18.960796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:18.960796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.960796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:18.961415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.961415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:18.961415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:18.961415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:18.961415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:18.961415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:18.961415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:18.961415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:03:18.961415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:18.961415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:18.961415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.961415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:18.961979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:18.962590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:18.962590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:18.962590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.962590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:18.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:18.963675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:18.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:18.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:18.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:18.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:18.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:18.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.964481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:18.967053 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:18.967485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:18.967657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:18.967949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:18.968418 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:18.968743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.969607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:18.969607 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:18.969607 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:18.969607 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:18.969607 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:18.969607 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:18.969607 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.969607 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:18.970326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:18.970750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:18.970750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:18.970750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:18.970750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:18.970750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:18.970750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:18.970750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:18.970750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.970750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:18.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:18.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:18.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:18.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:18.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:18.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:18.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:18.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:18.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:18.971490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:18.971871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:18.971871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:18.971871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.971871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:18.972320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:18.972778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:18.973376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:18.973376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:18.973376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.973376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:18.994649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:18.995116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:18.995332 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:18.995555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:18.996131 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:18.996475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:18.996680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.997402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:18.997402 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:18.997402 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:18.997402 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:18.997402 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:18.997402 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:18.997402 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.997402 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:18.998652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.998652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:18.998652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:18.998652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:18.998652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:18.998652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:18.998652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:18.998652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.998652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:18.998652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:18.998652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:18.998652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:18.999162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:18.999162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:18.999162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:18.999162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:18.999162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:18.999162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:18.999162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:18.999162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:18.999162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:18.999162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:18.999162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:18.999162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:18.999162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:18.999743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:18.999743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:19.000132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:19.000795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:19.001638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:19.001638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:19.001638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:21.230821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:21.231489 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:21.231795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:21.233041 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:21.266693 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:21.267062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:21.267959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:21.267959 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:21.267959 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:21.267959 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:21.267959 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:21.267959 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:21.267959 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:21.267959 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:21.268952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:21.268952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:21.268952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:21.268952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:21.268952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:21.268952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:21.268952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:21.268952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:21.268952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:21.268952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:21.268952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:21.268952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:21.269691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:21.269691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:21.269691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:21.269691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:21.269691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:21.269691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:21.269691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:21.269691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:21.269691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:21.269691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:21.269691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:21.269691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:21.269691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:21.270445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:21.271272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:21.272309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:21.272309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:21.272309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:21.272309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:21.272309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:21.272309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:23.927953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:23.928812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:23.928812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:23.929700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:23.930023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:23.930301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.931169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:23.931169 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:23.931169 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:23.931169 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:23.931169 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:23.931169 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:23.931169 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.931169 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:23.932682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.932682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:23.932682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:23.932682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:23.932682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:23.932682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:23.932682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:23.932682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.932682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:23.932682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:23.932682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:23.932682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.933097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:23.933097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:23.933097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:23.933097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:23.933097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:23.933097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:23.933097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:23.933097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:23.933097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:23.933097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:23.933097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:23.933097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:23.933097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:23.933462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.933462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:23.934485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:23.934941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:23.935467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:23.935467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.935467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:23.957858 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:23.958229 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:23.958448 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:23.958635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:03:23.959134 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:23.959455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.960238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:23.960238 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:23.960238 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:23.960238 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:23.960238 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:23.960238 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:23.960238 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.960238 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:23.961429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:23.962359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:23.962359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:23.962359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:23.962359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:23.962359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:23.962359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:23.962359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:23.962359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.962359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:23.962801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.962801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:23.962801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:23.962801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:23.962801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:23.962801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:23.962801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:23.962801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:03:23.962801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:23.962801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:23.962801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.962801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:23.963394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:23.964192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:23.964192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:23.964192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.964192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:23.964423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:23.965239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:23.966171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:23.966171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:23.966171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:23.966171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:23.966171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:23.966171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.966171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:23.967524 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:23.967812 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:23.967915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:23.968033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:23.968492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:23.968903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.969599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:23.969599 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:23.969599 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:23.969599 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:23.969599 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:23.969599 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:23.969599 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.969599 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:23.970313 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:23.970744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:23.970744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:23.970744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:23.970744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:23.970744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:23.970744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:23.970744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:23.970744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.970744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:23.971132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.971132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:23.971132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:23.971132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:23.971132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:23.971132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:23.971132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:23.971132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:23.971132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:23.971132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.971132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:23.971506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:23.972016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:23.972016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:23.972016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.972016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:23.972169 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:23.972843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:23.973516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:23.973516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:23.973516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.973516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:23.995685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:23.996182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:23.996383 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:23.996627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:23.997048 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:23.997416 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:23.997635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.998348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:23.998348 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:23.998348 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:23.998348 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:23.998348 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:23.998348 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:23.998348 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.998348 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:23.999618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:23.999618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:23.999618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:23.999618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:23.999618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:23.999618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:23.999618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:23.999618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.999618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:23.999618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:23.999618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:23.999618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:23.999980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:23.999980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:23.999980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:23.999980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:23.999980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:23.999980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:23.999980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:23.999980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:23.999980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:23.999980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:23.999980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:23.999980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:23.999980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:24.000351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:24.000351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:24.001117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:24.001641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:24.002151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:24.002151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:24.002151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:26.231601 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:26.232454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:26.232454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:26.233550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:26.267964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:26.268225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:26.268920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:26.268920 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:26.268920 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:26.268920 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:26.268920 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:26.268920 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:26.268920 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:26.268920 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:26.269728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:26.269728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:26.269728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:26.269728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:26.269728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:26.269728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:26.269728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:26.269728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:26.269728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:26.269728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:26.269728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:26.269728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:26.270409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:26.270409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:26.270409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:26.270409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:26.270409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:26.270409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:26.270409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:26.270409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:26.270409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:26.270409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:26.270409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:26.270409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:26.270409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:26.270953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:26.271656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:26.272485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:26.272485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:26.272485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:26.272485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:26.272485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:26.272485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:28.928633 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:28.929492 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:28.929708 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:28.930682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:28.930973 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:28.931197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.932078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:28.932078 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:28.932078 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:28.932078 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:28.932078 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:28.932078 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:28.932078 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.932078 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:28.933870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.933870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:28.933870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:28.933870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:28.933870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:28.933870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:28.933870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:28.933870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.933870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:28.933870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:28.933870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:28.933870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.934715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:28.934715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:28.934715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:28.934715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:28.934715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:28.934715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:28.934715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:28.934715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:28.934715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:28.934715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:28.934715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:28.934715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:28.934715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:28.935554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.935554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:28.936255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:28.937030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:28.937530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:28.937530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.937530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:28.958234 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:28.958493 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:28.958644 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:28.958816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:03:28.959319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:28.959673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.960562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:28.960562 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:28.960562 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:28.960562 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:28.960562 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:28.960562 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:28.960562 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.960562 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:28.961953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:28.963187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:28.963187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:28.963187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:28.963187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:28.963187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:28.963187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:28.963187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:28.963187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.963187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:28.963870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.963870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:28.963870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:28.963870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:28.963870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:28.963870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:28.963870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:28.963870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:03:28.963870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:28.963870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:28.963870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.963870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:28.964787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:28.965433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:28.965433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:28.965433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.965433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:28.965816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:28.966319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:28.966768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:28.966768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:28.966768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:28.966768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:28.966768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:28.966768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.966768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:28.967892 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:28.968225 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:28.968372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:28.968544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:28.968981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:28.969406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.970071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:28.970071 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:28.970071 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:28.970071 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:28.970071 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:28.970071 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:28.970071 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.970071 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:28.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:28.971550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:28.971550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:28.971550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:28.971550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:28.971550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:28.971550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:28.971550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:28.971550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.971550 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:28.972172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.972172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:28.972172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:28.972172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:28.972172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:28.972172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:28.972172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:28.972172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:28.972172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:28.972172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.972172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:28.972663 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:28.973226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:28.973226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:28.973226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.973226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:28.973813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:28.974375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:28.974860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:28.974860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:28.974860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.974860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:28.995994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:28.996299 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:28.996459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:28.996649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:28.997064 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:28.997490 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:28.997707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.998390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:28.998390 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:28.998390 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:28.998390 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:28.998390 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:28.998390 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:28.998390 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.998390 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:28.999351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:28.999351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:28.999351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:28.999351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:28.999351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:28.999351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:28.999351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:28.999351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.999351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:28.999351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:28.999351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:28.999351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:28.999845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:28.999845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:28.999845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:28.999845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:28.999845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:28.999845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:28.999845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:28.999845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:28.999845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:28.999845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:28.999845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:28.999845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:28.999845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:29.000422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:29.000422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:29.000745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:29.001256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:29.001729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:29.001729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:29.001729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:31.232130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:31.232862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:31.232862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:31.233025 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:31.268990 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:31.269515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:31.270854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:31.270854 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:31.270854 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:31.270854 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:31.270854 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:31.270854 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:31.270854 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:31.270854 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:31.272072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:31.272072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:31.272072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:31.272072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:31.272072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:31.272072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:31.272072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:31.272072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:31.272072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:31.272072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:31.272072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:31.272072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:31.272907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:31.272907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:31.272907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:31.272907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:31.272907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:31.272907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:31.272907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:31.272907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:31.272907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:31.272907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:31.272907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:31.272907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:31.272907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:31.273693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:31.274604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:31.275540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:31.275540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:31.275540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:31.275540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:31.275540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:31.275540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:33.929597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:33.930272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:33.930272 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:33.931150 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:33.931381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:33.931535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.932189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:33.932189 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:33.932189 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:33.932189 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:33.932189 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:33.932189 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:33.932189 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.932189 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:33.933713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.933713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:33.933713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:33.933713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:33.933713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:33.933713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:33.933713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:33.933713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.933713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:33.933713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:33.933713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:33.933713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.934280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:33.934280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:33.934280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:33.934280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:33.934280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:33.934280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:33.934280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:33.934280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:33.934280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:33.934280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:33.934280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:33.934280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:33.934280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:33.935069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.935069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:33.935666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:33.936375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:33.937176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:33.937176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.937176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:33.958786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:33.959230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:33.959417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:33.959597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:03:33.960362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:33.960362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.961168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:33.961168 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:33.961168 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:33.961168 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:33.961168 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:33.961168 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:33.961168 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.961168 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:33.962574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:33.963248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:33.963248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:33.963248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:33.963248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:33.963248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:33.963248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:33.963248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:33.963248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.963248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:33.963857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.963857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:33.963857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:33.963857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:33.963857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:33.963857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:33.963857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:33.963857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:03:33.963857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:33.963857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:33.963857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.963857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:33.964416 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:33.965048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:33.965048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:33.965048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.965048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:33.965455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:33.966190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:33.966980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:33.966980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:33.966980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:33.966980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:33.966980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:33.966980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.966980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:33.968346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:33.968721 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:33.968948 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:33.969161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:33.969626 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:33.970009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.970678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:33.970678 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:33.970678 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:33.970678 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:33.970678 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:33.970678 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:33.970678 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.970678 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:33.971448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:33.971838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:33.971838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:33.971838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:33.971838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:33.971838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:33.971838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:33.971838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:33.971838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.971838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:33.972564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.972564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:33.972564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:33.972564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:33.972564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:33.972564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:33.972564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:33.972564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:33.972564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:33.972564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.972564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:33.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:33.973274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:33.973274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:33.973274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.973274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:33.973675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:33.974138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:33.974655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:33.974655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:33.974655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.974655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:33.996509 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:33.996837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:33.996954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:33.997133 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:33.997606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:33.997989 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:33.998199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.998882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:33.998882 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:33.998882 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:33.998882 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:33.998882 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:33.998882 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:33.998882 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.998882 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:33.999742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:33.999742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:33.999742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:33.999742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:33.999742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:33.999742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:33.999742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:33.999742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:33.999742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:33.999742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:33.999742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:33.999742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:34.000052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:34.000052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:34.000052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:34.000052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:34.000052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:34.000052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:34.000052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:34.000052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:34.000052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:34.000052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:34.000052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:34.000052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:34.000052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:34.000419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:34.000419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:34.000749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:34.001147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:34.001652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:34.001652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:34.001652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:36.232838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:36.233798 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:36.233879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:36.235032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:36.272166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:36.272773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:36.273818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:36.273818 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:36.273818 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:36.273818 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:36.273818 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:36.273818 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:36.273818 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:36.273818 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:36.274735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:36.274735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:36.274735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:36.274735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:36.274735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:36.274735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:36.274735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:36.274735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:36.274735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:36.274735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:36.274735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:36.274735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:36.275466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:36.275466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:36.275466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:36.275466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:36.275466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:36.275466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:36.275466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:36.275466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:36.275466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:36.275466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:36.275466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:36.275466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:36.275466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:36.276075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:36.276876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:36.277876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:36.277876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:36.277876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:36.277876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:36.277876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:36.277876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:38.930253 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:38.931103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:38.931103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:38.932389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:38.932797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:38.933092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.934296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:38.934296 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:38.934296 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:38.934296 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:38.934296 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:38.934296 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:38.934296 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.934296 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:38.935934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.935934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:38.935934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:38.935934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:38.935934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:38.935934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:38.935934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:38.935934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.935934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:38.935934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:38.935934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:38.935934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.936563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:38.936563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:38.936563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:38.936563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:38.936563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:38.936563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:38.936563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:38.936563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:38.936563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:38.936563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:38.936563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:38.936563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:38.936563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:38.937117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.937117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:38.937676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:38.938414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:38.939285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:38.939285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.939285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:38.959386 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:38.959718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:38.959836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:38.959958 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:03:38.960331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:38.960506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.960946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:38.960946 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:38.960946 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:38.960946 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:38.960946 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:38.960946 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:38.960946 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.960946 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:38.961683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:38.962264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:38.962264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:38.962264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:38.962264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:38.962264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:38.962264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:38.962264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:38.962264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.962264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:38.962625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.962625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:38.962625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:38.962625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:38.962625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:38.962625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:38.962625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:38.962625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:03:38.962625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:38.962625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:38.962625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.962625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:38.963069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:38.963596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:38.964048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:38.964732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:38.964732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:38.964732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:38.964732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:38.964732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:38.964732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:38.964732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:38.964732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:38.964732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:38.964732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:38.964732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.964732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:38.968851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:38.969118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:38.969252 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:38.969382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:38.969872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:38.970314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:38.971144 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:38.971144 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:38.971144 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:38.971144 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:38.971144 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:38.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.971144 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:38.972136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:38.972636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:38.972636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:38.972636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:38.972636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:38.972636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:38.972636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:38.972636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:38.972636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.972636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:38.973618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.973618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:38.973618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:38.973618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:38.973618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:38.973618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:38.973618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:38.973618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:38.973618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:38.973618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.973618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:38.974032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:38.974727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:38.975491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:38.976500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:38.976500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:38.976500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:38.976500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:38.976500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:38.976500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:38.976500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:38.976500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:38.976500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:38.976500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.976500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:38.997130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:38.997638 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:38.997737 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:38.997945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:38.998302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:38.998535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:38.998676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.999168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:38.999168 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:38.999168 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:38.999168 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:38.999168 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:38.999168 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:38.999168 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:38.999168 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:39.000257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:39.000257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:39.000257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:39.000257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:39.000257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:39.000257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:39.000257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:39.000257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:39.000257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:39.000257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:39.000257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:39.000257 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:39.001263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:39.001263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:39.001263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:39.001263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:39.001263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:39.001263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:39.001263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:39.001263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:39.001263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:39.001263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:39.001263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:39.001263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:39.001263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:39.001901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:39.002614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:39.003479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:39.003479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:39.003479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:39.003479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:39.003479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:39.003479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:41.234036 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:41.234697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:41.235142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:41.236071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:41.271752 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:41.271995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:41.272839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:41.272839 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:41.272839 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:41.272839 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:41.272839 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:41.272839 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:41.272839 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:41.272839 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:41.273611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:41.273611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:41.273611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:41.273611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:41.273611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:41.273611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:41.273611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:41.273611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:41.273611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:41.273611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:41.273611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:41.273611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:41.274280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:41.274280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:41.274280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:41.274280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:41.274280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:41.274280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:41.274280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:41.274280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:41.274280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:41.274280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:41.274280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:41.274280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:41.274280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:41.274938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:41.275774 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:41.276719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:41.276719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:41.276719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:41.276719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:41.276719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:41.276719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:43.931006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:43.931676 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:43.931676 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:43.932614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:43.932927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:43.933162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.933942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:43.933942 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:43.933942 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:43.933942 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:43.933942 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:43.933942 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:43.933942 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.933942 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:43.935534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.935534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:43.935534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:43.935534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:43.935534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:43.935534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:43.935534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:43.935534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.935534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:43.935534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:43.935534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:43.935534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.936126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:43.936126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:43.936126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:43.936126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:43.936126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:43.936126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:43.936126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:43.936126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:43.936126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:43.936126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:43.936126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:43.936126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:43.936126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:43.936724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.936724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:43.937942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:43.938891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:43.939728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:43.939728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.939728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:43.960001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:43.960440 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:43.960653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:43.960845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:03:43.961498 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:43.961826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.962693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:43.962693 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:43.962693 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:43.962693 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:43.962693 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:43.962693 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:43.962693 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.962693 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:43.963882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:43.964626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:43.964626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:43.964626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:43.964626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:43.964626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:43.964626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:43.964626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:43.964626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.964626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:43.965143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.965143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:43.965143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:43.965143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:43.965143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:43.965143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:43.965143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:43.965143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:03:43.965143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:43.965143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:43.965143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.965143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:43.965707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:43.966370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:43.966370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:43.966370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.966370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:43.966850 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:43.967617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:43.968505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:43.968505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:43.968505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:43.968505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:43.968505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:43.968505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.968505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:43.969335 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:43.969627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:43.969728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:43.969859 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:43.970319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:43.970615 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.971115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:43.971115 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:43.971115 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:43.971115 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:43.971115 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:43.971115 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:43.971115 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.971115 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:43.971887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:43.972272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:43.972272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:43.972272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:43.972272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:43.972272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:43.972272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:43.972272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:43.972272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.972272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:43.972976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.972976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:43.972976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:43.972976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:43.972976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:43.972976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:43.972976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:43.972976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:43.972976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:43.972976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.972976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:43.973296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:43.973659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:43.973659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:43.973659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.973659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:43.974115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:43.974606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:43.975142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:43.975142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:43.975142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:43.975142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:43.997706 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:43.998137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:43.998359 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:43.998692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:43.999223 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:43.999506 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:43.999656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:44.000359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:44.000359 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:44.000359 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:44.000359 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:44.000359 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:44.000359 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:44.000359 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:44.000359 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:44.001752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:44.001752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:44.001752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:44.001752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:44.001752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:44.001752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:44.001752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:44.001752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:44.001752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:44.001752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:44.001752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:44.001752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:44.002337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:44.002337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:44.002337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:44.002337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:44.002337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:44.002337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:44.002337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:44.002337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:44.002337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:44.002337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:44.002337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:44.002337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:44.002337 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:44.002998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:44.002998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:44.003398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:44.004132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:44.005472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:44.005472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:44.005472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:46.234894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:46.235645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:46.235645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:46.236833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:46.269710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:46.270314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:46.271411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:46.271411 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:46.271411 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:46.271411 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:46.271411 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:46.271411 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:46.271411 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:46.271411 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:46.272447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:46.272447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:46.272447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:46.272447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:46.272447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:46.272447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:46.272447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:46.272447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:46.272447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:46.272447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:46.272447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:46.272447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:46.273044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:46.273044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:46.273044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:46.273044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:46.273044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:46.273044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:46.273044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:46.273044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:46.273044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:46.273044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:46.273044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:46.273044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:46.273044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:46.273630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:46.274355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:46.275178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:46.275178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:46.275178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:46.275178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:46.275178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:46.275178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:48.931682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:48.932950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:48.932950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:48.934029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:48.934511 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:48.934837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.936019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:48.936019 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:48.936019 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:48.936019 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:48.936019 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:48.936019 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:48.936019 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.936019 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:48.937633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.937633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:48.937633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:48.937633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:48.937633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:48.937633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:48.937633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:48.937633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.937633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:48.937633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:48.937633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:48.937633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.938230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:48.938230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:48.938230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:48.938230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:48.938230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:48.938230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:48.938230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:48.938230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:48.938230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:48.938230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:48.938230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:48.938230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:48.938230 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:48.938687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.938687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:48.939263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:48.939833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:48.940517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:48.940517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.940517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:48.960496 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:48.961024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:48.961243 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:48.961516 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:03:48.962254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:48.962661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.963616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:48.963616 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:48.963616 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:48.963616 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:48.963616 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:48.963616 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:48.963616 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.963616 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:48.965154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:48.966161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:48.966161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:48.966161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:48.966161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:48.966161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:48.966161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:48.966161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:48.966161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.966161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:48.966649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.966649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:48.966649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:48.966649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:48.966649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:48.966649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:48.966649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:48.966649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:03:48.966649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:48.966649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:48.966649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.966649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:48.967269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:48.967884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:48.967884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:48.967884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.967884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:48.968606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:48.969464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:48.970038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:48.970038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:48.970038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:48.970038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:48.970038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:48.970038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.970038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:48.970038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:48.971588 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:48.971768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:48.971969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:48.972613 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:48.972953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.973553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:48.973553 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:48.973553 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:48.973553 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:48.973553 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:48.973553 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:48.973553 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.973553 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:48.974101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:48.974640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:48.974640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:48.974640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:48.974640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:48.974640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:48.974640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:48.974640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:48.974640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.974640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:48.974908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.974908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:48.974908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:48.974908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:48.974908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:48.974908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:48.974908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:48.974908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:48.974908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:48.974908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.974908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:48.975310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:48.975773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:48.976232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:48.976801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:48.976801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:48.976801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:48.976801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:48.976801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:48.976801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:48.976801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:48.976801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:48.976801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:48.976801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:48.976801 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:48.998376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:48.998696 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:48.998957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:48.999083 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:48.999730 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:49.000346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:49.000455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:49.001402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:49.001402 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:49.001402 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:49.001402 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:49.001402 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:49.001402 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:49.001402 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:49.001402 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:49.002640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:49.002640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:49.002640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:49.002640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:49.002640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:49.002640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:49.002640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:49.002640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:49.002640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:49.002640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:49.002640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:49.002640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:49.003338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:49.003338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:49.003338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:49.003338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:49.003338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:49.003338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:49.003338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:49.003338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:49.003338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:49.003338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:49.003338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:49.003338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:49.003338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:49.003952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:49.003952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:49.004151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:49.005029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:49.005797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:49.005797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:49.005797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:51.235382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:51.236266 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:51.236529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:51.237985 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:51.273755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:51.274196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:51.275140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:51.275140 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:51.275140 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:51.275140 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:51.275140 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:51.275140 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:51.275140 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:51.275140 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:51.276374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:51.276374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:51.276374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:51.276374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:51.276374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:51.276374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:51.276374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:51.276374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:51.276374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:51.276374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:51.276374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:51.276374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:51.276963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:51.276963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:51.276963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:51.276963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:51.276963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:51.276963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:51.276963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:51.276963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:51.276963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:51.276963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:51.276963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:51.276963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:51.276963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:51.277540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:51.278248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:51.279332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:51.279332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:51.279332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:51.279332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:51.279332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:51.279332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:53.932515 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:53.933444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:53.933444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:53.934345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:53.934711 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:53.934968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.936078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:53.936078 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:53.936078 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:53.936078 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:53.936078 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:53.936078 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:53.936078 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.936078 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:53.937751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.937751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:53.937751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:53.937751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:53.937751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:53.937751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:53.937751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:53.937751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.937751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:53.937751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:53.937751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:53.937751 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.938781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:53.938781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:53.938781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:53.938781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:53.938781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:53.938781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:53.938781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:53.938781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:53.938781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:53.938781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:53.938781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:53.938781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:53.938781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:53.939754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:53.940642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:53.942466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:53.942466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:53.942466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:53.942466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:53.942466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.942466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:53.960956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:53.961307 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:53.961475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:53.961688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:03:53.962163 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:53.962494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.963247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:53.963247 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:53.963247 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:53.963247 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:53.963247 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:53.963247 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:53.963247 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.963247 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:53.964168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:53.965977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:53.967427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:53.967427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:53.967427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:03:53.967427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:53.967427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:53.967427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.967427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:53.967427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:53.967427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:53.967427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.967427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:53.967427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:53.967427 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:53.968020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:53.968020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:53.968020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:53.968020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:53.968020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:53.968020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:53.968020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:53.968020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:53.968020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:53.968020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:53.968020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.968020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:53.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:53.970381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:53.971256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:53.971256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:53.971256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:53.971256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:53.971256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:53.971256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.971256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:53.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:53.971751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:53.971837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:53.971950 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:53.972442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:53.972795 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.973497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:53.973497 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:53.973497 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:53.973497 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:53.973497 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:53.973497 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:53.973497 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.973497 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:53.974243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:53.974863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:53.974863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:53.974863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:53.974863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:53.974863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:53.974863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:53.974863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:53.974863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.974863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:53.975243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.975243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:53.975243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:53.975243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:53.975243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:53.975243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:53.975243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:53.975243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:53.975243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:53.975243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.975243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:53.975816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:53.976369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:53.976841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:53.977173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:53.977173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:53.977173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:53.977173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:53.977173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:53.977173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:53.977173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:53.977173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:53.977173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:53.977173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:53.977173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:53.998682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:53.998959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:53.999123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:53.999502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:54.000015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:54.000449 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:54.000694 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:54.001553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:54.001553 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:54.001553 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:54.001553 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:54.001553 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:54.001553 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:54.001553 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:54.001553 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:54.003149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:54.003149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:54.003149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:54.003149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:54.003149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:54.003149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:54.003149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:54.003149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:54.003149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:54.003149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:54.003149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:54.003149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:54.004012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:54.004012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:54.004012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:54.004012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:54.004012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:54.004012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:54.004012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:54.004012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:54.004012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:54.004012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:54.004012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:54.004012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:54.004012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:54.004548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:54.004548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:54.005174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:54.005838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:54.006631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:54.006631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:54.006631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:56.236063 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:56.236865 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:56.237040 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:56.238159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:56.271917 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:56.272342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:56.273247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:56.273247 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:56.273247 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:56.273247 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:56.273247 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:56.273247 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:56.273247 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:56.273247 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:56.274324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:56.274324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:56.274324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:56.274324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:56.274324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:56.274324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:56.274324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:56.274324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:56.274324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:56.274324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:56.274324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:56.274324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:56.274893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:56.274893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:56.274893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:56.274893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:56.274893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:56.274893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:56.274893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:56.274893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:56.274893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:56.274893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:56.274893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:56.274893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:56.274893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:56.275472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:56.276078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:56.276875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:56.276875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:56.276875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:56.276875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:56.276875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:56.276875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:58.933070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:58.933864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:58.933864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:58.934713 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:58.935050 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:58.935332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.936186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:58.936186 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:58.936186 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:58.936186 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:58.936186 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:58.936186 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:58.936186 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.936186 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:58.937759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.937759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:58.937759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:58.937759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:58.937759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:58.937759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:58.937759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:58.937759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.937759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:58.937759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:58.937759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:58.937759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.938345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:58.938345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:58.938345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:58.938345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:58.938345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:58.938345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:58.938345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:58.938345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:58.938345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:58.938345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:58.938345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:58.938345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:58.938345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:58.938895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.938895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:58.939438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:58.940130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:58.940926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:58.940926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.940926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:58.961392 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:58.961828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:58.962082 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:58.962447 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:03:58.962997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:58.963342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.964248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:58.964248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:58.964248 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:58.964248 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:58.964248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:58.964248 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:58.964248 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.964248 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:58.965425 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:58.966095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:58.966095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:58.966095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:58.966095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:58.966095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:58.966095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:58.966095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:58.966095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.966095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:58.966975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.966975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:58.966975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:58.966975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:58.966975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:58.966975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:58.966975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:58.966975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:03:58.966975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:58.966975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:58.966975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.966975 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:58.967513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:58.968124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:58.968124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:58.968124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.968124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:58.968865 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:58.969650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:58.970198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:58.970198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:58.970198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:58.970198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:58.970198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:58.970198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.970198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:58.971849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:58.972062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:58.972147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:58.972265 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:58.972786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:58.973235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.973953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:58.973953 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:58.973953 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:58.973953 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:58.973953 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:58.973953 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:58.973953 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.973953 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:03:58.974516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:03:58.974921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:58.974921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:03:58.974921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:58.974921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:03:58.974921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:03:58.974921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:58.974921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:03:58.974921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.974921 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:03:58.975163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.975163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:58.975163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:58.975163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:58.975163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:58.975163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:58.975163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:58.975163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:58.975163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:58.975163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.975163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:58.975518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:58.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:58.976290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:58.976821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:58.976821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:58.976821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:58.976821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:58.976821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:58.976821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:58.976821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:58.976821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:58.976821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:58.976821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:58.976821 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:03:58.999107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:03:58.999643 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:03:58.999920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:03:59.000473 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:03:59.000953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:59.001324 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:03:59.001552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:59.002360 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:03:59.002360 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:59.002360 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:03:59.002360 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:03:59.002360 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:59.002360 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:03:59.002360 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:59.002360 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:03:59.003888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:59.003888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:03:59.003888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:59.003888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:03:59.003888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:03:59.003888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:59.003888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:59.003888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:59.003888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:59.003888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:59.003888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:03:59.003888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:59.004243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:03:59.004243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:59.004243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:03:59.004243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:59.004243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:03:59.004243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:03:59.004243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:59.004243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:03:59.004243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:59.004243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:03:59.004243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:03:59.004243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:59.004243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:03:59.004598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:59.004598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:03:59.005990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:03:59.006450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:03:59.006951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:03:59.006951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:03:59.006951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:01.236867 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:01.240252 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:01.240252 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:01.240252 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:01.276303 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:01.276680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:01.277545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:01.277545 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:01.277545 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:01.277545 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:01.277545 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:01.277545 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:01.277545 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:01.277545 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:01.279076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:01.279076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:01.279076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:01.279076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:01.279076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:01.279076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:01.279076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:01.279076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:01.279076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:01.279076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:01.279076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:01.279076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:01.279730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:01.279730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:01.279730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:01.279730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:01.279730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:01.279730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:01.279730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:01.279730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:01.279730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:01.279730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:01.279730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:01.279730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:01.279730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:01.280319 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:01.281036 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:01.282132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:01.282132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:01.282132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:01.282132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:01.282132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:01.282132 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:03.933818 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:03.934843 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:03.935001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:03.936504 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:03.936918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:03.937269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.938317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:03.938317 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:03.938317 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:03.938317 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:03.938317 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:03.938317 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:03.938317 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.938317 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:03.940380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.940380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:03.940380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:03.940380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:03.940380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:03.940380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:03.940380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:03.940380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.940380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:03.940380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:03.940380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:03.940380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.940893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:03.940893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:03.940893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:03.940893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:03.940893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:03.940893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:03.940893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:03.940893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:03.940893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:03.940893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:03.940893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:03.940893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:03.940893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:03.941299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.941299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:03.941738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:03.942349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:03.942890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:03.942890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.942890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:03.961955 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:03.962953 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:03.963185 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:03.963485 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:04:03.964001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:03.964225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.964882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:03.964882 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:03.964882 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:03.964882 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:03.964882 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:03.964882 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:03.964882 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.964882 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:03.966485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:03.967275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:03.967275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:03.967275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:03.967275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:03.967275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:03.967275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:03.967275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:03.967275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.967275 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:03.968034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.968034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:03.968034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:03.968034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:03.968034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:03.968034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:03.968034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:03.968034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:04:03.968034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:03.968034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:03.968034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.968034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:03.968624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:03.969147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:03.969147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:03.969147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.969147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:03.969838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:03.970302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:03.970761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:03.970761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:03.970761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:03.970761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:03.970761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:03.970761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.970761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:03.972381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:03.972852 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:03.973052 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:03.973344 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:03.973806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:03.974037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.974867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:03.974867 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:03.974867 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:03.974867 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:03.974867 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:03.974867 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:03.974867 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.974867 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:03.975778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:03.976247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:03.976247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:03.976247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:03.976247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:03.976247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:03.976247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:03.976247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:03.976247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.976247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:03.976668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.976668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:03.976668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:03.976668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:03.976668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:03.976668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:03.976668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:03.976668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:03.976668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:03.976668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.976668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:03.976984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:03.977388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:03.977388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:03.977388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.977388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:03.977733 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:03.978330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:03.978864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:03.978864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:03.978864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:03.978864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:03.999867 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:04.000354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:04.000534 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:04.000786 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:04.001337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:04.001702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:04.001930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:04.002825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:04.002825 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:04.002825 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:04.002825 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:04.002825 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:04.002825 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:04.002825 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:04.002825 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:04.004234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:04.004234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:04.004234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:04.004234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:04.004234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:04.004234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:04.004234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:04.004234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:04.004234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:04.004234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:04.004234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:04.004234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:04.004776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:04.004776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:04.004776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:04.004776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:04.004776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:04.004776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:04.004776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:04.004776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:04.004776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:04.004776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:04.004776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:04.004776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:04.004776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:04.005334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:04.005334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:04.005782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:04.006512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:04.007286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:04.007286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:04.007286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:06.238158 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:06.238947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:06.238947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:06.240068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:06.275070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:06.275485 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:06.276323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:06.276323 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:06.276323 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:06.276323 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:06.276323 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:06.276323 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:06.276323 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:06.276323 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:06.277061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:06.277061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:06.277061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:06.277061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:06.277061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:06.277061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:06.277061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:06.277061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:06.277061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:06.277061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:06.277061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:06.277061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:06.277668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:06.277668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:06.277668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:06.277668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:06.277668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:06.277668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:06.277668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:06.277668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:06.277668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:06.277668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:06.277668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:06.277668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:06.277668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:06.278239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:06.279156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:06.280102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:06.280102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:06.280102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:06.280102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:06.280102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:06.280102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:08.934572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:08.935247 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:08.935454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:08.936825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:08.937173 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:08.937455 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.938355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:08.938355 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:08.938355 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:08.938355 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:08.938355 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:08.938355 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:08.938355 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.938355 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:08.939957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.939957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:08.939957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:08.939957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:08.939957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:08.939957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:08.939957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:08.939957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.939957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:08.939957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:08.939957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:08.939957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.940696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:08.940696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:08.940696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:08.940696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:08.940696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:08.940696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:08.940696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:08.940696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:08.940696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:08.940696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:08.940696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:08.940696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:08.940696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:08.941410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.941410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:08.942075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:08.942995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:08.944341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:08.944341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.944341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:08.962682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:08.963059 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:08.963275 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:08.963486 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:04:08.963985 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:08.964314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.965114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:08.965114 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:08.965114 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:08.965114 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:08.965114 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:08.965114 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:08.965114 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.965114 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:08.966581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:08.967465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:08.967465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:08.967465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:08.967465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:08.967465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:08.967465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:08.967465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:08.967465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.967465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:08.967909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.967909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:08.967909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:08.967909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:08.967909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:08.967909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:08.967909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:08.967909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:04:08.967909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:08.967909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:08.967909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.967909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:08.968519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:08.969572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:08.970242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:08.971307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:08.971307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:08.971307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:08.971307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:08.971307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:08.971307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:08.971307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:08.971307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:08.971307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:08.971307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:08.971307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.971307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:08.972848 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:08.973116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:08.973216 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:08.973380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:08.973955 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:08.974162 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.974866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:08.974866 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:08.974866 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:08.974866 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:08.974866 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:08.974866 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:08.974866 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.974866 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:08.975681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:08.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:08.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:08.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:08.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:08.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:08.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:08.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:08.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:08.976547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.976547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:08.976547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:08.976547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:08.976547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:08.976547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:08.976547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:08.976547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:08.976547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:08.976547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.976547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:08.976886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:08.977390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:08.977390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:08.977390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.977390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:08.977563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:08.978073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:08.978701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:08.978701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:08.978701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:08.978701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:09.000375 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:09.000746 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:09.000923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:09.001200 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:09.001686 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:09.002029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:09.002302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:09.003030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:09.003030 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:09.003030 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:09.003030 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:09.003030 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:09.003030 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:09.003030 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:09.003030 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:09.004422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:09.004422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:09.004422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:09.004422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:09.004422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:09.004422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:09.004422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:09.004422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:09.004422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:09.004422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:09.004422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:09.004422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:09.005072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:09.005072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:09.005072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:09.005072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:09.005072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:09.005072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:09.005072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:09.005072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:09.005072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:09.005072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:09.005072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:09.005072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:09.005072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:09.005950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:09.005950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:09.006100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:09.006945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:09.007566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:09.007566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:09.007566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:11.238650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:11.239369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:11.239369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:11.240201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:11.253459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 16:04:11.277503 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:11.277963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:11.278832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:11.278832 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:11.278832 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:11.278832 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:11.278832 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:11.278832 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:11.278832 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:11.278832 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:11.279639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:11.279639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:11.279639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:11.279639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:11.279639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:11.279639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:11.279639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:11.279639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:11.279639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:11.279639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:11.279639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:11.279639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:11.280179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:11.280179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:11.280179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:11.280179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:11.280179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:11.280179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:11.280179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:11.280179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:11.280179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:11.280179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:11.280179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:11.280179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:11.280179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:11.281256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:11.282426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:11.283446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:11.283446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:11.283446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:11.283446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:11.283446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:11.283446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:13.935261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:13.936196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:13.936196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:13.936900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:13.937241 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:13.937481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.938351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:13.938351 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:13.938351 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:13.938351 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:13.938351 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:13.938351 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:13.938351 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.938351 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:13.940277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.940277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:13.940277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:13.940277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:13.940277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:13.940277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:13.940277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:13.940277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.940277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:13.940277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:13.940277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:13.940277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.941264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:13.941264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:13.941264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:13.941264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:13.941264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:13.941264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:13.941264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:13.941264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:13.941264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:13.941264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:13.941264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:13.941264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:13.941264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:13.941942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.941942 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:13.942235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:13.943090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:13.943815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:13.943815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.943815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:13.963119 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:13.963544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:13.963734 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:13.963937 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:04:13.964463 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:13.964744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.965474 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:13.965474 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:13.965474 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:13.965474 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:13.965474 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:13.965474 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:13.965474 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.965474 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:13.966681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:13.967444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:13.967444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:13.967444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:13.967444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:13.967444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:13.967444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:13.967444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:13.967444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.967444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:13.968181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.968181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:13.968181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:13.968181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:13.968181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:13.968181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:13.968181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:13.968181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:04:13.968181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:13.968181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:13.968181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.968181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:13.968830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:13.969503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:13.969503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:13.969503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.969503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:13.969814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:13.970548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:13.971509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:13.971509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:13.971509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:13.971509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:13.971509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:13.971509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.971509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:13.973235 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:13.973510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:13.973615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:13.973753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:13.974276 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:13.974747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.975530 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:13.975530 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:13.975530 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:13.975530 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:13.975530 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:13.975530 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:13.975530 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.975530 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:13.976294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:13.976736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:13.976736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:13.976736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:13.976736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:13.976736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:13.976736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:13.976736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:13.976736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.976736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:13.977224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.977224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:13.977224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:13.977224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:13.977224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:13.977224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:13.977224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:13.977224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:13.977224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:13.977224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.977224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:13.977582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:13.977969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:13.977969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:13.977969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.977969 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:13.978458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:13.978978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:13.979514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:13.979514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:13.979514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:13.979514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:14.000833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:14.001225 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:14.001438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:14.001707 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:14.002284 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:14.002924 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:14.003253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:14.004159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:14.004159 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:14.004159 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:14.004159 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:14.004159 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:14.004159 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:14.004159 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:14.004159 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:14.005890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:14.005890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:14.005890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:14.005890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:14.005890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:14.005890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:14.005890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:14.005890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:14.005890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:14.005890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:14.005890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:14.005890 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:14.006492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:14.006492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:14.006492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:14.006492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:14.006492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:14.006492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:14.006492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:14.006492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:14.006492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:14.006492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:14.006492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:14.006492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:14.006492 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:14.007064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:14.007064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:14.007861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:14.008435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:14.008998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:14.008998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:14.008998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:16.239407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:16.240199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:16.240199 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:16.240705 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:16.270359 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:16.270719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:16.271576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:16.271576 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:16.271576 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:16.271576 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:16.271576 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:16.271576 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:16.271576 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:16.271576 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:16.272322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:16.272322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:16.272322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:16.272322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:16.272322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:16.272322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:16.272322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:16.272322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:16.272322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:16.272322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:16.272322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:16.272322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:16.272898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:16.272898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:16.272898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:16.272898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:16.272898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:16.272898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:16.272898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:16.272898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:16.272898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:16.272898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:16.272898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:16.272898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:16.272898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:16.273493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:16.275008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:16.276682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:16.276682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:16.276682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:16.276682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:16.276682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:16.276682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:18.935947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:18.936792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:18.937045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:18.938145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:18.938411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:18.938612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.939452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:18.939452 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:18.939452 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:18.939452 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:18.939452 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:18.939452 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:18.939452 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.939452 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:18.941533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.941533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:18.941533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:18.941533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:18.941533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:18.941533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:18.941533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:18.941533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.941533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:18.941533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:18.941533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:18.941533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.942133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:18.942133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:18.942133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:18.942133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:18.942133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:18.942133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:18.942133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:18.942133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:18.942133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:18.942133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:18.942133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:18.942133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:18.942133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:18.942671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.942671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:18.944041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:18.944898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:18.945632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:18.945632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.945632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:18.963996 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:18.964625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:18.964845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:18.964845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:04:18.965566 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:18.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.966664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:18.966664 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:18.966664 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:18.966664 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:18.966664 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:18.966664 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:18.966664 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.966664 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:18.967608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:18.970280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:18.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:18.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:18.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:04:18.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:18.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:18.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:18.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:18.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:18.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:18.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:18.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:18.972882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:18.972882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:18.972882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:18.972882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:18.972882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:18.972882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:18.972882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:18.972882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:18.972882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:18.972882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:18.972882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.972882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:18.975847 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:18.976365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:18.976888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:18.976888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:18.976888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:18.976888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:18.976888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:18.976888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.976888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:18.976888 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:18.976888 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:18.976888 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:18.977235 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:18.977235 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:18.977235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.977235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:18.977235 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:18.977235 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:18.977235 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:18.977235 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:18.977235 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:18.977235 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.977235 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:18.977235 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:18.977655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:18.978183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:18.978183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:18.978183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:18.978183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:18.978183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.978183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:18.978183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.978183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:18.978183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:18.978183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:18.978183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:18.978183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:18.978183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:18.978714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:18.978714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:18.978714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.978714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:18.978714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:18.978714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:18.978714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.978714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:18.978714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:18.978714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:18.978714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:18.978714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:18.978714 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:18.979124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:18.979124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:18.979124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:18.979124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:18.979124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:18.979124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:18.979124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:18.979124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.979124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:18.980101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:18.980734 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:18.981356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:18.981356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:18.981356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:18.981356 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:19.001382 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:19.002033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:19.002224 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:19.002512 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:19.003087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:19.003615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:19.003849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:19.004839 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:19.004839 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:19.004839 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:19.004839 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:19.004839 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:19.004839 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:19.004839 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:19.004839 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:19.006419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:19.006419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:19.006419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:19.006419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:19.006419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:19.006419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:19.006419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:19.006419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:19.006419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:19.006419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:19.006419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:19.006419 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:19.007259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:19.007259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:19.007259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:19.007259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:19.007259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:19.007259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:19.007259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:19.007259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:19.007259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:19.007259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:19.007259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:19.007259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:19.007259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:19.008047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:19.008047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:19.009039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:19.009717 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:19.010303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:19.010303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:19.010303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:21.240096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:21.240811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:21.240811 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:21.241528 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:21.273816 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:21.274179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:21.274984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:21.274984 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:21.274984 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:21.274984 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:21.274984 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:21.274984 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:21.274984 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:21.274984 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:21.275739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:21.275739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:21.275739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:21.275739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:21.275739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:21.275739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:21.275739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:21.275739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:21.275739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:21.275739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:21.275739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:21.275739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:21.276391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:21.276391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:21.276391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:21.276391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:21.276391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:21.276391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:21.276391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:21.276391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:21.276391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:21.276391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:21.276391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:21.276391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:21.276391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:21.277062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:21.277834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:21.278887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:21.278887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:21.278887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:21.278887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:21.278887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:21.278887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:23.936648 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:23.937483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:23.937555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:23.938679 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:23.939062 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:23.939399 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.940559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:23.940559 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:23.940559 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:23.940559 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:23.940559 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:23.940559 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:23.940559 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.940559 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:23.942178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.942178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:23.942178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:23.942178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:23.942178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:23.942178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:23.942178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:23.942178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.942178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:23.942178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:23.942178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:23.942178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.942792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:23.942792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:23.942792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:23.942792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:23.942792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:23.942792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:23.942792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:23.942792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:23.942792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:23.942792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:23.942792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:23.942792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:23.942792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:23.943441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.943441 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:23.944021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:23.944744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:23.945374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:23.945374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.945374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:23.964594 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:23.965158 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:23.965373 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:23.965601 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:04:23.966129 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:23.966532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.967391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:23.967391 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:23.967391 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:23.967391 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:23.967391 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:23.967391 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:23.967391 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.967391 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:23.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:23.969496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:23.969496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:23.969496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:23.969496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:23.969496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:23.969496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:23.969496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:23.969496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.969496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:23.970136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.970136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:23.970136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:23.970136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:23.970136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:23.970136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:23.970136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:23.970136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:04:23.970136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:23.970136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:23.970136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.970136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:23.970718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:23.971317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:23.971317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:23.971317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.971317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:23.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:23.972691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:23.973479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:23.973479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:23.973479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:23.973479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:23.973479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:23.973479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.973479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:23.974300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:23.974695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:23.974788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:23.974923 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:23.975575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:23.976113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.977053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:23.977053 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:23.977053 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:23.977053 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:23.977053 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:23.977053 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:23.977053 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.977053 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:23.977749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:23.978158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:23.978158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:23.978158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:23.978158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:23.978158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:23.978158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:23.978158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:23.978158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.978158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:23.978432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.978432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:23.978432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:23.978432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:23.978432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:23.978432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:23.978432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:23.978432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:23.978432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:23.978432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.978432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:23.978754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:23.979142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:23.979142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:23.979142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.979142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:23.979294 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:23.979953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:23.980696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:23.980696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:23.980696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:23.980696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:24.001903 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:24.002487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:24.002663 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:24.002921 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:24.003451 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:24.003853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:24.004097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:24.005037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:24.005037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:24.005037 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:24.005037 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:24.005037 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:24.005037 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:24.005037 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:24.005037 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:24.006565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:24.006565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:24.006565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:24.006565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:24.006565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:24.006565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:24.006565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:24.006565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:24.006565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:24.006565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:24.006565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:24.006565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:24.007248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:24.007248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:24.007248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:24.007248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:24.007248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:24.007248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:24.007248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:24.007248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:24.007248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:24.007248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:24.007248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:24.007248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:24.007248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:24.007840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:24.007840 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:24.008421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:24.009172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:24.009971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:24.009971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:24.009971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:26.241018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:26.241874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:26.241874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:26.243345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:26.281362 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:26.281693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:26.282436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:26.282436 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:26.282436 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:26.282436 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:26.282436 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:26.282436 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:26.282436 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:26.282436 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:26.283116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:26.283116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:26.283116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:26.283116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:26.283116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:26.283116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:26.283116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:26.283116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:26.283116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:26.283116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:26.283116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:26.283116 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:26.284146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:26.284146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:26.284146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:26.284146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:26.284146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:26.284146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:26.284146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:26.284146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:26.284146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:26.284146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:26.284146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:26.284146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:26.284146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:26.284991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:26.285988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:26.287905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:26.287905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:26.287905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:26.287905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:26.287905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:26.287905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:28.937301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:28.938281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:28.938281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:28.939436 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:28.939636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:28.939759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.940394 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:28.940394 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:28.940394 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:28.940394 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:28.940394 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:28.940394 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:28.940394 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.940394 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:28.941902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.941902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:28.941902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:28.941902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:28.941902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:28.941902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:28.941902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:28.941902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.941902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:28.941902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:28.941902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:28.941902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.942768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:28.942768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:28.942768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:28.942768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:28.942768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:28.942768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:28.942768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:28.942768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:28.942768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:28.942768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:28.942768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:28.942768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:28.942768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:28.943744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:28.944434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:28.945250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:28.945250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:28.945250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:28.945250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:28.945250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.945250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:28.965113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:28.965595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:28.965794 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:28.966087 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:04:28.966532 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:28.966676 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.967281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:28.967281 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:28.967281 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:28.967281 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:28.967281 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:28.967281 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:28.967281 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.967281 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:28.968739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:28.969881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:28.969881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:28.969881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:28.969881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:28.969881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:28.969881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:28.969881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:28.969881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.969881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:28.970625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.970625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:28.970625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:28.970625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:28.970625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:28.970625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:28.970625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:28.970625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:04:28.970625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:28.970625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:28.970625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.970625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:28.971558 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:28.972486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:28.973509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:28.974835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:28.974835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:28.974835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:28.974835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:28.974835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:28.974835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:28.974835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:28.974835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:28.974835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:28.974835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:28.974835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.974835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:28.974835 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:28.975393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:28.975393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:28.975501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:28.975748 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:28.975909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.976321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:28.976321 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:28.976321 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:28.976321 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:28.976321 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:28.976321 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:28.976321 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.976321 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:28.977128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:28.977653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:28.977653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:28.977653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:28.977653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:28.977653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:28.977653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:28.977653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:28.977653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.977653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:28.978280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.978280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:28.978280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:28.978280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:28.978280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:28.978280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:28.978280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:28.978280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:28.978280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:28.978280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.978280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:28.978741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:28.979309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:28.979309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:28.979309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.979309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:28.979496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:28.980180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:28.980956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:28.980956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:28.980956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:28.980956 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:29.002487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:29.002791 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:29.002899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:29.003095 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:29.003356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:29.003510 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:29.003607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:29.004063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:29.004063 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:29.004063 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:29.004063 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:29.004063 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:29.004063 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:29.004063 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:29.004063 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:29.005070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:29.005070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:29.005070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:29.005070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:29.005070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:29.005070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:29.005070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:29.005070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:29.005070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:29.005070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:29.005070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:29.005070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:29.005876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:29.005876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:29.005876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:29.005876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:29.005876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:29.005876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:29.005876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:29.005876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:29.005876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:29.005876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:29.005876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:29.005876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:29.005876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:29.006782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:29.007788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:29.009004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:29.009004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:29.009004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:29.009004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:29.009004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:29.009004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:31.241599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:31.242432 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:31.242432 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:31.243846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:31.282081 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:31.282503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:31.283369 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:31.283369 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:31.283369 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:31.283369 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:31.283369 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:31.283369 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:31.283369 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:31.283369 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:31.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:31.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:31.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:31.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:31.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:31.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:31.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:31.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:31.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:31.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:31.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:31.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:31.284928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:31.284928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:31.284928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:31.284928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:31.284928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:31.284928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:31.284928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:31.284928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:31.284928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:31.284928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:31.284928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:31.284928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:31.284928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:31.285494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:31.286153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:31.286937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:31.286937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:31.286937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:31.286937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:31.286937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:31.286937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:33.938319 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:33.938987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:33.939190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:33.940296 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:33.940557 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:33.940730 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.941483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:33.941483 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:33.941483 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:33.941483 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:33.941483 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:33.941483 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:33.941483 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.941483 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:33.943029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.943029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:33.943029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:33.943029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:33.943029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:33.943029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:33.943029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:33.943029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.943029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:33.943029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:33.943029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:33.943029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.943560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:33.943560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:33.943560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:33.943560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:33.943560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:33.943560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:33.943560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:33.943560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:33.943560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:33.943560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:33.943560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:33.943560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:33.943560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:33.944120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.944120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:33.945542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:33.946513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:33.947465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:33.947465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.947465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:33.965692 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:33.966116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:33.966344 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:33.966537 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:04:33.967288 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:33.967386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.968108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:33.968108 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:33.968108 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:33.968108 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:33.968108 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:33.968108 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:33.968108 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.968108 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:33.969430 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:33.969886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:33.969886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:33.969886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:33.969886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:33.969886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:33.969886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:33.969886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:33.969886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.969886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:33.970825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.970825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:33.970825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:33.970825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:33.970825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:33.970825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:33.970825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:33.970825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:04:33.970825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:33.970825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:33.970825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.970825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:33.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:33.971628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:33.971628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:33.971628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.971628 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:33.972545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:33.972987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:33.973465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:33.973465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:33.973465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:33.973465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:33.973465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:33.973465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.973465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:33.975340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:33.975674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:33.975802 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:33.975931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:33.976356 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:33.976732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.977418 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:33.977418 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:33.977418 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:33.977418 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:33.977418 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:33.977418 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:33.977418 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.977418 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:33.978138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:33.978561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:33.978561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:33.978561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:33.978561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:33.978561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:33.978561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:33.978561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:33.978561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.978561 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:33.979056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.979056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:33.979056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:33.979056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:33.979056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:33.979056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:33.979056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:33.979056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:33.979056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:33.979056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.979056 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:33.979405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:33.979788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:33.979788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:33.979788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.979788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:33.980129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:33.980618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:33.981124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:33.981124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:33.981124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:33.981124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:34.003041 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:34.003501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:34.003688 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:34.003944 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:34.004415 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:34.004773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:34.005016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:34.005764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:34.005764 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:34.005764 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:34.005764 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:34.005764 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:34.005764 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:34.005764 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:34.005764 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:34.007105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:34.007105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:34.007105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:34.007105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:34.007105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:34.007105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:34.007105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:34.007105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:34.007105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:34.007105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:34.007105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:34.007105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:34.007708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:34.007708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:34.007708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:34.007708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:34.007708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:34.007708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:34.007708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:34.007708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:34.007708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:34.007708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:34.007708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:34.007708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:34.007708 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:34.008262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:34.008262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:34.008892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:34.009826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:34.010702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:34.010702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:34.010702 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:36.242298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:36.243182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:36.243182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:36.244417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:36.278187 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:36.278423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:36.278979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:36.278979 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:36.278979 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:36.278979 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:36.278979 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:36.278979 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:36.278979 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:36.278979 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:36.279909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:36.279909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:36.279909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:36.279909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:36.279909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:36.279909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:36.279909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:36.279909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:36.279909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:36.279909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:36.279909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:36.279909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:36.280925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:36.280925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:36.280925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:36.280925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:36.280925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:36.280925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:36.280925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:36.280925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:36.280925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:36.280925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:36.280925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:36.280925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:36.280925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:36.281741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:36.282509 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:36.283547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:36.283547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:36.283547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:36.283547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:36.283547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:36.283547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:38.938862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:38.939658 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:38.939658 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:38.940675 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:38.940981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:38.941234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.942043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:38.942043 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:38.942043 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:38.942043 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:38.942043 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:38.942043 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:38.942043 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.942043 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:38.943553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.943553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:38.943553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:38.943553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:38.943553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:38.943553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:38.943553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:38.943553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.943553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:38.943553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:38.943553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:38.943553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.944072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:38.944072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:38.944072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:38.944072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:38.944072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:38.944072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:38.944072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:38.944072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:38.944072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:38.944072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:38.944072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:38.944072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:38.944072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:38.945546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:38.945960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:38.946458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:38.946458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:38.946458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:38.946458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:38.946458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.946458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:38.966132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:38.966517 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:38.966801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:38.966895 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:04:38.967592 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:38.968326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.968914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:38.968914 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:38.968914 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:38.968914 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:38.968914 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:38.968914 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:38.968914 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.968914 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:38.969935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:38.970627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:38.970627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:38.970627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:38.970627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:38.970627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:38.970627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:38.970627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:38.970627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.970627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:38.971192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.971192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:38.971192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:38.971192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:38.971192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:38.971192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:38.971192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:38.971192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:04:38.971192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:38.971192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:38.971192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.971192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:38.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:38.972385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:38.972385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:38.972385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.972385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:38.972808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:38.973545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:38.974377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:38.974377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:38.974377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:38.974377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:38.974377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:38.974377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.974377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:38.975784 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:38.976096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:38.976227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:38.976363 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:38.976850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:38.977231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.977914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:38.977914 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:38.977914 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:38.977914 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:38.977914 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:38.977914 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:38.977914 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.977914 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:38.978684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:38.979304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:38.979304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:38.979304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:38.979304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:38.979304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:38.979304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:38.979304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:38.979304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.979304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:38.979585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.979585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:38.979585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:38.979585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:38.979585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:38.979585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:38.979585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:38.979585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:38.979585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:38.979585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.979585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:38.979903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:38.980297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:38.980297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:38.980297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.980297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:38.980678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:38.981131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:38.981645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:38.981645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:38.981645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:38.981645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:39.003469 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:39.004331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:39.004499 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:39.004740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:39.005424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:39.005796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:39.006053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:39.006904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:39.006904 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:39.006904 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:39.006904 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:39.006904 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:39.006904 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:39.006904 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:39.006904 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:39.008220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:39.008220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:39.008220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:39.008220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:39.008220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:39.008220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:39.008220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:39.008220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:39.008220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:39.008220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:39.008220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:39.008220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:39.008726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:39.008726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:39.008726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:39.008726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:39.008726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:39.008726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:39.008726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:39.008726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:39.008726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:39.008726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:39.008726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:39.008726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:39.008726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:39.009264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:39.009264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:39.009758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:39.010472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:39.011309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:39.011309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:39.011309 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:41.243187 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:41.244035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:41.244035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:41.244898 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:41.278845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:41.279143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:41.279895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:41.279895 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:41.279895 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:41.279895 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:41.279895 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:41.279895 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:41.279895 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:41.279895 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:41.280666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:41.280666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:41.280666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:41.280666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:41.280666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:41.280666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:41.280666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:41.280666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:41.280666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:41.280666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:41.280666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:41.280666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:41.281119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:41.281119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:41.281119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:41.281119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:41.281119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:41.281119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:41.281119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:41.281119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:41.281119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:41.281119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:41.281119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:41.281119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:41.281119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:41.281648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:41.282289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:41.282970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:41.282970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:41.282970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:41.282970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:41.282970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:41.282970 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:43.939517 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:43.940186 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:43.940424 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:43.941377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:43.941630 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:43.941793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.942462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:43.942462 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:43.942462 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:43.942462 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:43.942462 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:43.942462 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:43.942462 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.942462 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:43.943996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.943996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:43.943996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:43.943996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:43.943996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:43.943996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:43.943996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:43.943996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.943996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:43.943996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:43.943996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:43.943996 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.944750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:43.944750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:43.944750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:43.944750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:43.944750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:43.944750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:43.944750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:43.944750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:43.944750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:43.944750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:43.944750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:43.944750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:43.944750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:43.945410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.945410 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:43.945677 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:43.946571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:43.947487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:43.947487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.947487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:43.966730 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:43.967113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:43.967341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:43.967525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:04:43.968098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:43.968473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.969471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:43.969471 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:43.969471 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:43.969471 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:43.969471 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:43.969471 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:43.969471 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.969471 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:43.970552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:43.972887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:43.973600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:43.973600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:43.973600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:04:43.973600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:43.973600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:43.973600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.973600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:43.973600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:43.973600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:43.973600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.973600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:43.973600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:43.973600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:43.974112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:43.974112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:43.974112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:43.974112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:43.974112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:43.974112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:43.974112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:43.974112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:43.974112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:43.974112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:43.974112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.974112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:43.974621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:43.975251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:43.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:43.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:43.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:43.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:43.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:43.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.975901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:43.976539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:43.977032 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:43.977147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:43.977303 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:43.977897 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:43.978380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:43.979429 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:43.979429 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:43.979429 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:43.979429 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:43.979429 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:43.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:43.980189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:43.980748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:43.980748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:43.980748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:43.980748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:43.980748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:43.980748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:43.980748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:43.980748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.980748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:43.981140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.981140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:43.981140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:43.981140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:43.981140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:43.981140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:43.981140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:43.981140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:43.981140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:43.981140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.981140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:43.981484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:43.982006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:43.982006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:43.982006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.982006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:43.982841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:43.983344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:43.983845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:43.983845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:43.983845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:43.983845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:44.004882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:44.005369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:44.005541 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:44.005797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:44.006355 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:44.006789 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:44.007075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:44.008077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:44.008077 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:44.008077 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:44.008077 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:44.008077 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:44.008077 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:44.008077 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:44.008077 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:44.009846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:44.009846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:44.009846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:44.009846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:44.009846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:44.009846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:44.009846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:44.009846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:44.009846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:44.009846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:44.009846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:44.009846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:44.010183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:44.010183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:44.010183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:44.010183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:44.010183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:44.010183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:44.010183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:44.010183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:44.010183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:44.010183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:44.010183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:44.010183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:44.010183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:44.010552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:44.010552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:44.011398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:44.011805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:44.012303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:44.012303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:44.012303 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:46.243886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:46.244718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:46.244718 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:46.246026 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:46.279668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:46.279807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:46.280349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:46.280349 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:46.280349 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:46.280349 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:46.280349 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:46.280349 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:46.280349 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:46.280349 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:46.281135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:46.281135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:46.281135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:46.281135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:46.281135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:46.281135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:46.281135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:46.281135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:46.281135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:46.281135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:46.281135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:46.281135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:46.281855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:46.281855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:46.281855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:46.281855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:46.281855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:46.281855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:46.281855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:46.281855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:46.281855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:46.281855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:46.281855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:46.281855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:46.281855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:46.282521 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:46.283346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:46.284357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:46.284357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:46.284357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:46.284357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:46.284357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:46.284357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:48.940261 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:48.941098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:48.941098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:48.942136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:48.942731 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:48.942945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.943949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:48.943949 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:48.943949 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:48.943949 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:48.943949 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:48.943949 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:48.943949 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.943949 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:48.945898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.945898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:48.945898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:48.945898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:48.945898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:48.945898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:48.945898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:48.945898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.945898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:48.945898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:48.945898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:48.945898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.946582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:48.946582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:48.946582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:48.946582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:48.946582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:48.946582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:48.946582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:48.946582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:48.946582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:48.946582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:48.946582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:48.946582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:48.946582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:48.947183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:48.947989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:48.948891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:48.948891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:48.948891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:48.948891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:48.948891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.948891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:48.967203 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:48.967713 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:48.967914 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:48.968192 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:04:48.968741 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:48.969052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.969913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:48.969913 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:48.969913 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:48.969913 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:48.969913 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:48.969913 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:48.969913 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.969913 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:48.971223 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:48.971757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:48.971757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:48.971757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:48.971757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:48.971757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:48.971757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:48.971757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:48.971757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.971757 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:48.972738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.972738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:48.972738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:48.972738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:48.972738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:48.972738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:48.972738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:48.972738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:04:48.972738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:48.972738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:48.972738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.972738 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:48.973290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:48.973693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:48.973693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:48.973693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.973693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:48.974476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:48.974928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:48.975435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:48.975435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:48.975435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:48.975435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:48.975435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:48.975435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.975435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:48.977113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:48.977417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:48.977549 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:48.977675 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:48.978126 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:48.978562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.979276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:48.979276 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:48.979276 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:48.979276 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:48.979276 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:48.979276 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:48.979276 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.979276 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:48.979964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:48.980444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:48.980444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:48.980444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:48.980444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:48.980444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:48.980444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:48.980444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:48.980444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.980444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:48.980817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.980817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:48.980817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:48.980817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:48.980817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:48.980817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:48.980817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:48.980817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:48.980817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:48.980817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.980817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:48.981150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:48.981555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:48.981555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:48.981555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.981555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:48.981948 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:48.982481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:48.982978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:48.982978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:48.982978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:48.982978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:49.005713 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:49.006168 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:49.006389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:49.006640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:49.007099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:49.007500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:49.007742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:49.009298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:49.009298 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:49.009298 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:49.009298 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:49.009298 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:49.009298 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:49.009298 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:49.009298 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:49.009772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:49.009772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:49.009772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:49.009772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:49.009772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:49.009772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:49.009772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:49.009772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:49.009772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:49.009772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:49.009772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:49.009772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:49.010433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:49.010433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:49.010433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:49.010433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:49.010433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:49.010433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:49.010433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:49.010433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:49.010433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:49.010433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:49.010433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:49.010433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:49.010433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:49.011157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:49.011866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:49.012895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:49.012895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:49.012895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:49.012895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:49.012895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:49.012895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:51.244428 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:51.245370 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:51.245370 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:51.246183 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:51.275615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:51.275973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:51.276783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:51.276783 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:51.276783 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:51.276783 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:51.276783 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:51.276783 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:51.276783 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:51.276783 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:51.277596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:51.277596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:51.277596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:51.277596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:51.277596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:51.277596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:51.277596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:51.277596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:51.277596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:51.277596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:51.277596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:51.277596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:51.278325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:51.278325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:51.278325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:51.278325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:51.278325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:51.278325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:51.278325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:51.278325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:51.278325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:51.278325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:51.278325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:51.278325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:51.278325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:51.279068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:51.279727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:51.280377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:51.280377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:51.280377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:51.280377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:51.280377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:51.280377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:53.941080 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:53.941866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:53.941866 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:53.942388 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:53.942633 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:53.942796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.943405 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:53.943405 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:53.943405 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:53.943405 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:53.943405 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:53.943405 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:53.943405 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.943405 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:53.944158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.944158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:53.944158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:53.944158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:53.944158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:53.944158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:53.944158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:53.944158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.944158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:53.944158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:53.944158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:53.944158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.945022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:53.945022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:53.945022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:53.945022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:53.945022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:53.945022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:53.945022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:53.945022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:53.945022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:53.945022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:53.945022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:53.945022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:53.945022 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:53.945643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:53.946372 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:53.947342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:53.947342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:53.947342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:53.947342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:53.947342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.947342 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:53.967689 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:53.968074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:53.968309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:53.968516 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:04:53.969226 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:53.969544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.970543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:53.970543 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:53.970543 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:53.970543 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:53.970543 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:53.970543 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:53.970543 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.970543 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:53.971792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:53.972361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:53.972361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:53.972361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:53.972361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:53.972361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:53.972361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:53.972361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:53.972361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.972361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:53.973176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.973176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:53.973176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:53.973176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:53.973176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:53.973176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:53.973176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:53.973176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:04:53.973176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:53.973176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:53.973176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.973176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:53.973607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:53.974001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:53.974001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:53.974001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.974001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:53.974979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:53.975500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:53.976065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:53.976065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:53.976065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:53.976065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:53.976065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:53.976065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.976065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:53.977564 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:53.977850 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:53.977958 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:53.978079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:53.978426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:53.978637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.979086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:53.979086 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:53.979086 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:53.979086 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:53.979086 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:53.979086 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:53.979086 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.979086 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:53.979800 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:53.980292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:53.980292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:53.980292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:53.980292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:53.980292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:53.980292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:53.980292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:53.980292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.980292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:53.980648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.980648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:53.980648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:53.980648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:53.980648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:53.980648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:53.980648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:53.980648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:53.980648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:53.980648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.980648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:53.981012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:53.981449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:53.981449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:53.981449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.981449 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:53.981861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:53.983111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:53.984015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:53.984015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:53.984015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:53.984015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:54.006225 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:54.006580 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:54.006751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:54.006980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:54.007461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:54.007796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:54.008014 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:54.008739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:54.008739 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:54.008739 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:54.008739 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:54.008739 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:54.008739 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:54.008739 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:54.008739 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:54.010501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:54.010501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:54.010501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:54.010501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:54.010501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:54.010501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:54.010501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:54.010501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:54.010501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:54.010501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:54.010501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:54.010501 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:54.011180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:54.011180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:54.011180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:54.011180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:54.011180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:54.011180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:54.011180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:54.011180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:54.011180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:54.011180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:54.011180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:54.011180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:54.011180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:54.011841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:54.011841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:54.012559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:54.013037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:54.013609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:54.013609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:54.013609 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:56.245094 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:56.246151 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:56.246151 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:56.247163 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:56.281463 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:56.281805 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:56.282555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:56.282555 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:56.282555 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:56.282555 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:56.282555 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:56.282555 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:56.282555 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:56.282555 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:56.283532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:56.283532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:56.283532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:56.283532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:56.283532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:56.283532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:56.283532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:56.283532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:56.283532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:56.283532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:56.283532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:56.283532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:56.284433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:56.284433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:56.284433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:56.284433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:56.284433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:56.284433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:56.284433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:56.284433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:56.284433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:56.284433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:56.284433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:56.284433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:56.284433 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:56.285060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:56.286776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:56.288017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:56.288017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:56.288017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:56.288017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:56.288017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:56.288017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:58.942907 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:58.943849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:58.943849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:58.944681 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:58.944904 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:58.945064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.945658 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:58.945658 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:58.945658 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:58.945658 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:58.945658 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:58.945658 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:58.945658 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.945658 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:58.946531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.946531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:58.946531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:58.946531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:58.946531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:58.946531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:58.946531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:58.946531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.946531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:58.946531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:58.946531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:58.946531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.947113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:58.947113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:58.947113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:58.947113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:58.947113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:58.947113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:58.947113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:58.947113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:58.947113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:58.947113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:58.947113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:58.947113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:58.947113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:58.947723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:58.948404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:58.949252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:58.949252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:58.949252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:58.949252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:58.949252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.949252 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:58.968323 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:58.968727 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:58.968946 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:58.969186 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:04:58.969857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:58.970181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:58.970924 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:58.970924 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:58.970924 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:58.970924 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:58.970924 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:58.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.970924 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:58.971798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:58.972531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:58.972531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:58.972531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:58.972531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:58.972531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:58.972531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:58.972531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:58.972531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.972531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:58.972966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.972966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:58.972966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:58.972966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:58.972966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:58.972966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:58.972966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:58.972966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:04:58.972966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:58.972966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:58.972966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.972966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:58.973540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:58.974110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:58.974110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:58.974110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.974110 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:58.974563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:58.975301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:58.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:58.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:58.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:58.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:58.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:58.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.976119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:58.977913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:58.978128 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:58.978309 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:58.978442 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:58.978883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:58.979348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.980041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:58.980041 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:58.980041 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:58.980041 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:58.980041 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:58.980041 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:58.980041 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.980041 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:04:58.980897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:04:58.981385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:58.981385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:04:58.981385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:58.981385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:04:58.981385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:04:58.981385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:58.981385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:04:58.981385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.981385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:04:58.982066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.982066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:58.982066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:58.982066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:58.982066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:58.982066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:58.982066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:58.982066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:58.982066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:58.982066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.982066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:58.982580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:58.982938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:58.982938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:58.982938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.982938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:58.983578 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:58.984079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:58.984654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:58.984654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:58.984654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:58.984654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:04:59.006798 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:04:59.007322 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:04:59.007623 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:04:59.007968 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:04:59.008540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:59.008936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:04:59.009177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:59.010065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:04:59.010065 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:59.010065 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:04:59.010065 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:04:59.010065 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:59.010065 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:04:59.010065 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:59.010065 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:04:59.011174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:59.011174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:04:59.011174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:59.011174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:04:59.011174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:04:59.011174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:59.011174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:59.011174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:59.011174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:59.011174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:59.011174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:04:59.011174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:59.011582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:04:59.011582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:59.011582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:04:59.011582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:59.011582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:04:59.011582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:04:59.011582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:59.011582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:04:59.011582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:59.011582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:04:59.011582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:04:59.011582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:59.011582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:04:59.011950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:59.011950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:04:59.012151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:04:59.012639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:04:59.013166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:04:59.013166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:04:59.013166 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:01.245679 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:01.246339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:01.246339 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:01.246848 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:01.276016 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:01.276323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:01.276994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:01.276994 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:01.276994 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:01.276994 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:01.276994 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:01.276994 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:01.276994 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:01.276994 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:01.277953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:01.277953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:01.277953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:01.277953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:01.277953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:01.277953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:01.277953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:01.277953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:01.277953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:01.277953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:01.277953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:01.277953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:01.278438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:01.278438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:01.278438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:01.278438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:01.278438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:01.278438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:01.278438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:01.278438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:01.278438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:01.278438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:01.278438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:01.278438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:01.278438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:01.278897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:01.278897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:01.279837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:01.280406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:01.280980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:01.280980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:01.280980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:03.944002 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:03.945027 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:03.945027 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:03.946191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:03.946476 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:03.946693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.947547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:03.947547 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:03.947547 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:03.947547 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:03.947547 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:03.947547 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:03.947547 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.947547 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:03.949019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.949019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:03.949019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:03.949019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:03.949019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:03.949019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:03.949019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:03.949019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.949019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:03.949019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:03.949019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:03.949019 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.949594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:03.949594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:03.949594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:03.949594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:03.949594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:03.949594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:03.949594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:03.949594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:03.949594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:03.949594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:03.949594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:03.949594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:03.949594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:03.950174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.950174 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:03.951007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:03.951470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:03.951968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:03.951968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.951968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:03.968972 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:03.969592 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:03.969778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:03.969962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:05:03.970599 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:03.970887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.971775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:03.971775 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:03.971775 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:03.971775 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:03.971775 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:03.971775 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:03.971775 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.971775 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:03.972962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:03.973756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:03.973756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:03.973756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:03.973756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:03.973756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:03.973756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:03.973756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:03.973756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.973756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:03.974582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.974582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:03.974582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:03.974582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:03.974582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:03.974582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:03.974582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:03.974582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:05:03.974582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:03.974582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:03.974582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.974582 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:03.975657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:03.976825 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:03.977831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:03.978661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:03.979061 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:03.979061 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:03.979629 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:03.979629 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.980122 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:03.980122 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:03.980122 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:03.980122 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:03.980122 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:03.980122 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:03.980122 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.980122 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:03.980900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:03.981818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:03.981818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:03.981818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:03.981818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:03.981818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:03.981818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:03.981818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:03.981818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.981818 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:03.982368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.982368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:03.982368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:03.982368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:03.982368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:03.982368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:03.982368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:03.982368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:03.982368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:03.982368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.982368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:03.983042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:03.983667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:03.984047 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:03.984560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:03.984560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:03.984560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:03.984560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:03.984560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:03.984560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:03.984560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:03.984560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:03.984560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:03.984560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:03.984560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:04.007456 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:04.007955 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:04.008089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:04.008342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:04.008872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:04.009005 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:04.009154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:04.009740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:04.009740 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:04.009740 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:04.009740 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:04.009740 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:04.009740 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:04.009740 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:04.009740 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:04.010650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:04.010650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:04.010650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:04.010650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:04.010650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:04.010650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:04.010650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:04.010650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:04.010650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:04.010650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:04.010650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:04.010650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:04.011011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:04.011011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:04.011011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:04.011011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:04.011011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:04.011011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:04.011011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:04.011011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:04.011011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:04.011011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:04.011011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:04.011011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:04.011011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:04.011388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:04.011388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:04.011666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:04.012123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:04.012679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:04.012679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:04.012679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:06.246345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:06.247450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:06.247450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:06.248548 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:06.281068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:06.281357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:06.281631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:06.281631 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:06.281631 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:06.281631 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:06.281631 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:06.281631 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:06.281631 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:06.281631 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:06.282476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:06.282476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:06.282476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:06.282476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:06.282476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:06.282476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:06.282476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:06.282476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:06.282476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:06.282476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:06.282476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:06.282476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:06.283289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:06.283289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:06.283289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:06.283289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:06.283289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:06.283289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:06.283289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:06.283289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:06.283289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:06.283289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:06.283289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:06.283289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:06.283289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:06.284359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:06.285153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:06.286092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:06.286092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:06.286092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:06.286092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:06.286092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:06.286092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:08.944720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:08.945851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:08.945851 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:08.947234 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:08.947637 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:08.947926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.949082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:08.949082 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:08.949082 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:08.949082 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:08.949082 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:08.949082 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:08.949082 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.949082 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:08.951316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.951316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:08.951316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:08.951316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:08.951316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:08.951316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:08.951316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:08.951316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.951316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:08.951316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:08.951316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:08.951316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.952109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:08.952109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:08.952109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:08.952109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:08.952109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:08.952109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:08.952109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:08.952109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:08.952109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:08.952109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:08.952109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:08.952109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:08.952109 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:08.953053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.953053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:08.953879 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:08.954450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:08.955009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:08.955009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.955009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:08.969515 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:08.969932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:08.970169 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:08.970517 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:05:08.971068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:08.971421 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.972239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:08.972239 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:08.972239 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:08.972239 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:08.972239 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:08.972239 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:08.972239 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.972239 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:08.973779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:08.975091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:08.975091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:08.975091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:08.975091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:08.975091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:08.975091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:08.975091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:08.975091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.975091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:08.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:08.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:08.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:08.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:08.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:08.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:08.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:05:08.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:08.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:08.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:08.976758 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:08.977812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:08.977812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:08.977812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.977812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:08.978039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:08.978586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:08.979396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:08.979396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:08.979396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:08.979396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:08.979396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:08.979396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.979396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:08.979396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:08.979396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:08.979396 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:08.979728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:08.979769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:08.980126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.980985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:08.980985 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:08.980985 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:08.980985 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:08.980985 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:08.980985 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:08.980985 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.980985 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:08.982102 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:08.982814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:08.982814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:08.982814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:08.982814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:08.982814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:08.982814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:08.982814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:08.982814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.982814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:08.983507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.983507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:08.983507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:08.983507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:08.983507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:08.983507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:08.983507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:08.983507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:08.983507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:08.983507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.983507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:08.984577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:08.985781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:08.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:08.986797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:08.986797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:08.986797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:08.986797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:08.986797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:08.986797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:08.986797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:08.986797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:08.986797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:08.986797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:08.986797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:09.008024 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:09.008597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:09.008762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:09.009015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:09.009679 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:09.010090 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:09.010478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:09.011238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:09.011238 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:09.011238 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:09.011238 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:09.011238 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:09.011238 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:09.011238 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:09.011238 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:09.012496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:09.012496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:09.012496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:09.012496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:09.012496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:09.012496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:09.012496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:09.012496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:09.012496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:09.012496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:09.012496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:09.012496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:09.013081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:09.013081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:09.013081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:09.013081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:09.013081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:09.013081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:09.013081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:09.013081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:09.013081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:09.013081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:09.013081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:09.013081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:09.013081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:09.013777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:09.013777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:09.014043 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:09.014631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:09.015238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:09.015238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:09.015238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:11.247060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:11.247856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:11.247856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:11.249103 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:11.260364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 16:05:11.280560 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:11.280803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:11.281423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:11.281423 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:11.281423 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:11.281423 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:11.281423 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:11.281423 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:11.281423 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:11.281423 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:11.282187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:11.282187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:11.282187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:11.282187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:11.282187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:11.282187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:11.282187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:11.282187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:11.282187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:11.282187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:11.282187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:11.282187 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:11.282654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:11.282654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:11.282654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:11.282654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:11.282654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:11.282654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:11.282654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:11.282654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:11.282654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:11.282654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:11.282654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:11.282654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:11.282654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:11.283139 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:11.283868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:11.284688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:11.284688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:11.284688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:11.284688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:11.284688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:11.284688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:13.945408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:13.945977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:13.946297 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:13.947170 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:13.947549 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:13.947797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.948667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:13.948667 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:13.948667 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:13.948667 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:13.948667 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:13.948667 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:13.948667 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.948667 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:13.950332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.950332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:13.950332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:13.950332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:13.950332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:13.950332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:13.950332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:13.950332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.950332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:13.950332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:13.950332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:13.950332 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.950878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:13.950878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:13.950878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:13.950878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:13.950878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:13.950878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:13.950878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:13.950878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:13.950878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:13.950878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:13.950878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:13.950878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:13.950878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:13.951435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.951435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:13.952624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:13.953487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:13.954305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:13.954305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.954305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:13.970029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:13.970419 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:13.970648 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:13.970887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:05:13.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:13.971939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.972820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:13.972820 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:13.972820 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:13.972820 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:13.972820 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:13.972820 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:13.972820 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.972820 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:13.974346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:13.974968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:13.974968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:13.974968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:13.974968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:13.974968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:13.974968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:13.974968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:13.974968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.974968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:13.976105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.976105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:13.976105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:13.976105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:13.976105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:13.976105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:13.976105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:13.976105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:05:13.976105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:13.976105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:13.976105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.976105 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:13.976656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:13.977231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:13.977231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:13.977231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.977231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:13.977572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:13.978308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:13.979039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:13.979039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:13.979039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:13.979039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:13.979039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:13.979039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.979039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:13.979381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:13.979487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:13.979649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:13.979872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:13.980326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:13.980655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.981363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:13.981363 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:13.981363 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:13.981363 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:13.981363 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:13.981363 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:13.981363 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.981363 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:13.982168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:13.982607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:13.982607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:13.982607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:13.982607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:13.982607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:13.982607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:13.982607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:13.982607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.982607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:13.983145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.983145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:13.983145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:13.983145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:13.983145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:13.983145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:13.983145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:13.983145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:13.983145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:13.983145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.983145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:13.983496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:13.983920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:13.983920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:13.983920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.983920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:13.984296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:13.984776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:13.985293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:13.985293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:13.985293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:13.985293 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:14.008625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:14.008900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:14.009004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:14.009171 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:14.009585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:14.009935 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:14.010357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:14.010991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:14.010991 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:14.010991 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:14.010991 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:14.010991 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:14.010991 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:14.010991 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:14.010991 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:14.011806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:14.011806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:14.011806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:14.011806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:14.011806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:14.011806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:14.011806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:14.011806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:14.011806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:14.011806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:14.011806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:14.011806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:14.012438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:14.012438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:14.012438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:14.012438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:14.012438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:14.012438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:14.012438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:14.012438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:14.012438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:14.012438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:14.012438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:14.012438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:14.012438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:14.012959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:14.013794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:14.014568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:14.014568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:14.014568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:14.014568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:14.014568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:14.014568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:16.248039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:16.249015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:16.249015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:16.250345 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:16.285015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:16.285183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:16.285639 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:16.285639 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:16.285639 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:16.285639 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:16.285639 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:16.285639 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:16.285639 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:16.285639 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:16.286420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:16.286420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:16.286420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:16.286420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:16.286420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:16.286420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:16.286420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:16.286420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:16.286420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:16.286420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:16.286420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:16.286420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:16.286799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:16.286799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:16.286799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:16.286799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:16.286799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:16.286799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:16.286799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:16.286799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:16.286799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:16.286799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:16.286799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:16.286799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:16.286799 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:16.287151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:16.287151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:16.287233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:16.287911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:16.288515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:16.288515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:16.288515 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:18.945952 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:18.946853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:18.946853 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:18.947848 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:18.948177 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:18.949066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.949513 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:18.949513 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:18.949513 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:18.949513 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:18.949513 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:18.949513 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:18.949513 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.949513 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:18.951436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.951436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:18.951436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:18.951436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:18.951436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:18.951436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:18.951436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:18.951436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.951436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:18.951436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:18.951436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:18.951436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.952953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:18.952953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:18.952953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:18.952953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:18.952953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:18.952953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:18.952953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:18.952953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:18.952953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:18.952953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:18.952953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:18.952953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:18.952953 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:18.953796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:18.954442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:18.955037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:18.955037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:18.955037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:18.955037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:18.955037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.955037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:18.970401 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:18.970624 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:18.970744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:18.970874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:05:18.971769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:18.972068 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.972977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:18.972977 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:18.972977 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:18.972977 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:18.972977 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:18.972977 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:18.972977 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.972977 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:18.974093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:18.974812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:18.974812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:18.974812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:18.974812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:18.974812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:18.974812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:18.974812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:18.974812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.974812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:18.975622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.975622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:18.975622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:18.975622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:18.975622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:18.975622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:18.975622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:18.975622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:05:18.975622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:18.975622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:18.975622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.975622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:18.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:18.978065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:18.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:18.979703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:18.979703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:18.979703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:18.979703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:18.979703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:18.979703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:18.979703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:18.979703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:18.979703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:18.979703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:18.979703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.979703 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:18.980581 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:18.980841 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:18.981037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:18.981191 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:18.981883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:18.982153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.982792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:18.982792 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:18.982792 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:18.982792 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:18.982792 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:18.982792 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:18.982792 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.982792 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:18.983570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:18.983885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:18.983885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:18.983885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:18.983885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:18.983885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:18.983885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:18.983885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:18.983885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.983885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:18.984548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.984548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:18.984548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:18.984548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:18.984548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:18.984548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:18.984548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:18.984548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:18.984548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:18.984548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.984548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:18.984989 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:18.985248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:18.985248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:18.985248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.985248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:18.985769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:18.986247 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:18.986581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:18.986581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:18.986581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:18.986581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:19.009466 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:19.009854 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:19.010008 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:19.010130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:19.010738 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:19.010981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:19.011129 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:19.011772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:19.011772 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:19.011772 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:19.011772 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:19.011772 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:19.011772 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:19.011772 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:19.011772 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:19.013236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:19.013236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:19.013236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:19.013236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:19.013236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:19.013236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:19.013236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:19.013236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:19.013236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:19.013236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:19.013236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:19.013236 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:19.013787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:19.013787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:19.013787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:19.013787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:19.013787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:19.013787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:19.013787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:19.013787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:19.013787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:19.013787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:19.013787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:19.013787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:19.013787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:19.014344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:19.014344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:19.015032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:19.015776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:19.017745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:19.017745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:19.017745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:21.248622 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:21.249258 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:21.249313 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:21.250488 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:21.281597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:21.281962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:21.282748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:21.282748 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:21.282748 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:21.282748 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:21.282748 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:21.282748 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:21.282748 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:21.282748 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:21.283659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:21.283659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:21.283659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:21.283659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:21.283659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:21.283659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:21.283659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:21.283659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:21.283659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:21.283659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:21.283659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:21.283659 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:21.284038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:21.284038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:21.284038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:21.284038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:21.284038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:21.284038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:21.284038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:21.284038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:21.284038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:21.284038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:21.284038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:21.284038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:21.284038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:21.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:21.284408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:21.284764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:21.285243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:21.285766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:21.285766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:21.285766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:23.946806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:23.947474 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:23.947674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:23.949301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:23.949301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:23.949301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.949662 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:23.949662 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:23.949662 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:23.949662 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:23.949662 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:23.949662 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:23.949662 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.949662 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:23.950995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.950995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:23.950995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:23.950995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:23.950995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:23.950995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:23.950995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:23.950995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.950995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:23.950995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:23.950995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:23.950995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.951552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:23.951552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:23.951552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:23.951552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:23.951552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:23.951552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:23.951552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:23.951552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:23.951552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:23.951552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:23.951552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:23.951552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:23.951552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:23.952172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.952172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:23.953348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:23.954149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:23.955064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:23.955064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.955064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:23.970932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:23.971221 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:23.971371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:23.971491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:05:23.972096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:23.972364 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.973086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:23.973086 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:23.973086 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:23.973086 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:23.973086 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:23.973086 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:23.973086 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.973086 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:23.974111 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:23.974869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:23.974869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:23.974869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:23.974869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:23.974869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:23.974869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:23.974869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:23.974869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.974869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:23.975397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.975397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:23.975397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:23.975397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:23.975397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:23.975397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:23.975397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:23.975397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:05:23.975397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:23.975397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:23.975397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.975397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:23.975907 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:23.976535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:23.976535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:23.976535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.976535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:23.977005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:23.977725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:23.978496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:23.978496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:23.978496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:23.978496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:23.978496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:23.978496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.978496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:23.981381 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:23.981657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:23.981754 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:23.981893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:23.982372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:23.982764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.983450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:23.983450 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:23.983450 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:23.983450 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:23.983450 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:23.983450 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:23.983450 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.983450 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:23.984219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:23.984642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:23.984642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:23.984642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:23.984642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:23.984642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:23.984642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:23.984642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:23.984642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.984642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:23.985220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.985220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:23.985220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:23.985220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:23.985220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:23.985220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:23.985220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:23.985220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:23.985220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:23.985220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.985220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:23.985563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:23.985959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:23.985959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:23.985959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.985959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:23.986937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:23.987459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:23.988222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:23.988222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:23.988222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:23.988222 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:24.010086 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:24.010502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:24.010670 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:24.010893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:24.011427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:24.011787 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:24.012030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:24.012769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:24.012769 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:24.012769 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:24.012769 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:24.012769 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:24.012769 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:24.012769 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:24.012769 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:24.014084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:24.014084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:24.014084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:24.014084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:24.014084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:24.014084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:24.014084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:24.014084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:24.014084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:24.014084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:24.014084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:24.014084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:24.014924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:24.014924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:24.014924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:24.014924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:24.014924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:24.014924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:24.014924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:24.014924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:24.014924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:24.014924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:24.014924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:24.014924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:24.014924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:24.015746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:24.016531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:24.017407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:24.017407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:24.017407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:24.017407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:24.017407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:24.017407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:26.249333 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:26.250137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:26.250137 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:26.251190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:26.285404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:26.285761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:26.286406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:26.286406 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:26.286406 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:26.286406 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:26.286406 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:26.286406 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:26.286406 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:26.286406 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:26.287124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:26.287124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:26.287124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:26.287124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:26.287124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:26.287124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:26.287124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:26.287124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:26.287124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:26.287124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:26.287124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:26.287124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:26.287924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:26.287924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:26.287924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:26.287924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:26.287924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:26.287924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:26.287924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:26.287924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:26.287924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:26.287924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:26.287924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:26.287924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:26.287924 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:26.288823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:26.289600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:26.290444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:26.290444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:26.290444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:26.290444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:26.290444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:26.290444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:28.947619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:28.948544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:28.948687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:28.949753 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:28.950101 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:28.950402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.951228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:28.951228 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:28.951228 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:28.951228 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:28.951228 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:28.951228 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:28.951228 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.951228 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:28.952746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.952746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:28.952746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:28.952746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:28.952746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:28.952746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:28.952746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:28.952746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.952746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:28.952746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:28.952746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:28.952746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.953380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:28.953380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:28.953380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:28.953380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:28.953380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:28.953380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:28.953380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:28.953380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:28.953380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:28.953380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:28.953380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:28.953380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:28.953380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:28.953973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.953973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:28.954653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:28.955442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:28.956324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:28.956324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.956324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:28.971757 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:28.974297 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:28.974297 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:28.974297 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:05:28.974297 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:28.974297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.974297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:28.974297 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:28.974297 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:28.974297 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:28.974896 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:28.974896 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:28.974896 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.974896 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:28.975424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:28.978078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:28.978798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:28.978798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:28.978798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:05:28.978798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:28.978798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:28.978798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.978798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:28.978798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:28.978798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:28.978798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.978798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:28.978798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:28.978798 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:28.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:28.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:28.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:28.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:28.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:28.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:28.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:28.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:28.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:28.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:28.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.979429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:28.979935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:28.980723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:28.981517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:28.981517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:28.981517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:28.981517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:28.981517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:28.981517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.981517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:28.981828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:28.982188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:28.982308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:28.982437 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:28.982862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:28.983228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.983884 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:28.983884 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:28.983884 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:28.983884 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:28.983884 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:28.983884 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:28.983884 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.983884 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:28.984665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:28.984987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:28.984987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:28.984987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:28.984987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:28.984987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:28.984987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:28.984987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:28.984987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.984987 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:28.985583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.985583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:28.985583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:28.985583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:28.985583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:28.985583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:28.985583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:28.985583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:28.985583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:28.985583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.985583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:28.985844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:28.986114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:28.986114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:28.986114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.986114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:28.986689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:28.987046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:28.987426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:28.987426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:28.987426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:28.987426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:29.010772 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:29.011320 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:29.011497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:29.011756 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:29.012182 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:29.012612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:29.012834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:29.013599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:29.013599 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:29.013599 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:29.013599 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:29.013599 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:29.013599 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:29.013599 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:29.013599 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:29.014968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:29.014968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:29.014968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:29.014968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:29.014968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:29.014968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:29.014968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:29.014968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:29.014968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:29.014968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:29.014968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:29.014968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:29.015256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:29.015256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:29.015256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:29.015256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:29.015256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:29.015256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:29.015256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:29.015256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:29.015256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:29.015256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:29.015256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:29.015256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:29.015256 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:29.015487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:29.015487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:29.016528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:29.016857 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:29.017260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:29.017260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:29.017260 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:31.249887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:31.250761 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:31.250761 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:31.251417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:31.285992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:31.286499 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:31.287587 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:31.287587 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:31.287587 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:31.287587 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:31.287587 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:31.287587 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:31.287587 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:31.287587 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:31.288647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:31.288647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:31.288647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:31.288647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:31.288647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:31.288647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:31.288647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:31.288647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:31.288647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:31.288647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:31.288647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:31.288647 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:31.289263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:31.289263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:31.289263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:31.289263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:31.289263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:31.289263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:31.289263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:31.289263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:31.289263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:31.289263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:31.289263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:31.289263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:31.289263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:31.289834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:31.290547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:31.291447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:31.291447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:31.291447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:31.291447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:31.291447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:31.291447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:33.948271 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:33.949041 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:33.949041 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:33.950066 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:33.950423 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:33.950654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.951806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:33.951806 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:33.951806 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:33.951806 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:33.951806 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:33.951806 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:33.951806 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.951806 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:33.953250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.953250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:33.953250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:33.953250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:33.953250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:33.953250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:33.953250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:33.953250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.953250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:33.953250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:33.953250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:33.953250 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.953902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:33.953902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:33.953902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:33.953902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:33.953902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:33.953902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:33.953902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:33.953902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:33.953902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:33.953902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:33.953902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:33.953902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:33.953902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:33.954464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.954464 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:33.954984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:33.955997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:33.956952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:33.956952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.956952 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:33.972099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:33.972464 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:33.972633 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:33.972813 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:05:33.973597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:33.973887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.974720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:33.974720 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:33.974720 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:33.974720 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:33.974720 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:33.974720 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:33.974720 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.974720 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:33.975892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:33.976545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:33.976545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:33.976545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:33.976545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:33.976545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:33.976545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:33.976545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:33.976545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.976545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:33.977189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.977189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:33.977189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:33.977189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:33.977189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:33.977189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:33.977189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:33.977189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:05:33.977189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:33.977189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:33.977189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.977189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:33.977793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:33.978415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:33.978415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:33.978415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.978415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:33.978728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:33.979176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:33.979706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:33.979706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:33.979706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:33.979706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:33.979706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:33.979706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.979706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:33.982160 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:33.982366 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:33.982478 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:33.982606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:33.982997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:33.983391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.984013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:33.984013 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:33.984013 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:33.984013 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:33.984013 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:33.984013 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:33.984013 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.984013 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:33.984909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:33.985361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:33.985361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:33.985361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:33.985361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:33.985361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:33.985361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:33.985361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:33.985361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.985361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:33.986030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.986030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:33.986030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:33.986030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:33.986030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:33.986030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:33.986030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:33.986030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:33.986030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:33.986030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.986030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:33.986391 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:33.986796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:33.986796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:33.986796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.986796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:33.987112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:33.987704 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:33.988244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:33.988244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:33.988244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:33.988244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:34.011184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:34.011552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:34.011709 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:34.011929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:34.012665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:34.013038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:34.013291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:34.014089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:34.014089 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:34.014089 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:34.014089 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:34.014089 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:34.014089 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:34.014089 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:34.014089 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:34.015366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:34.015366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:34.015366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:34.015366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:34.015366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:34.015366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:34.015366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:34.015366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:34.015366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:34.015366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:34.015366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:34.015366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:34.015861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:34.015861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:34.015861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:34.015861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:34.015861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:34.015861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:34.015861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:34.015861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:34.015861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:34.015861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:34.015861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:34.015861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:34.015861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:34.016366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:34.016366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:34.016860 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:34.017672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:34.018390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:34.018390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:34.018390 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:36.250715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:36.251453 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:36.251572 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:36.252406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:36.289879 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:36.290322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:36.291189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:36.291189 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:36.291189 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:36.291189 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:36.291189 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:36.291189 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:36.291189 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:36.291189 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:36.292350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:36.292350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:36.292350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:36.292350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:36.292350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:36.292350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:36.292350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:36.292350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:36.292350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:36.292350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:36.292350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:36.292350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:36.292749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:36.292749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:36.292749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:36.292749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:36.292749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:36.292749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:36.292749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:36.292749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:36.292749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:36.292749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:36.292749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:36.292749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:36.292749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:36.293172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:36.293172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:36.293461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:36.294029 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:36.294736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:36.294736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:36.294736 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:38.948874 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:38.949826 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:38.949826 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:38.950547 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:38.950731 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:38.950845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.951448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:38.951448 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:38.951448 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:38.951448 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:38.951448 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:38.951448 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:38.951448 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.951448 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:38.952914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.952914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:38.952914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:38.952914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:38.952914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:38.952914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:38.952914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:38.952914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.952914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:38.952914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:38.952914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:38.952914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.953669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:38.953669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:38.953669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:38.953669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:38.953669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:38.953669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:38.953669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:38.953669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:38.953669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:38.953669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:38.953669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:38.953669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:38.953669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:38.954091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.954091 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:38.954572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:38.955121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:38.955761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:38.955761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.955761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:38.972549 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:38.973105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:38.973105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:38.973298 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:05:38.973796 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:38.973963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.974535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:38.974535 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:38.974535 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:38.974535 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:38.974535 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:38.974535 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:38.974535 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.974535 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:38.975684 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:38.976435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:38.976435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:38.976435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:38.976435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:38.976435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:38.976435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:38.976435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:38.976435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.976435 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:38.976960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.976960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:38.976960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:38.976960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:38.976960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:38.976960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:38.976960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:38.976960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:05:38.976960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:38.976960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:38.976960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.976960 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:38.977589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:38.978178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:38.978178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:38.978178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.978178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:38.978580 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:38.979324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:38.980154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:38.980154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:38.980154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:38.980154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:38.980154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:38.980154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.980154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:38.982508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:38.982846 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:38.983015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:38.983197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:38.983567 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:38.983763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:38.984186 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:38.984186 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:38.984186 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:38.984186 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:38.984186 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:38.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:38.984946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:38.985409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:38.985409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:38.985409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:38.985409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:38.985409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:38.985409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:38.985409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:38.985409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.985409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:38.985888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.985888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:38.985888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:38.985888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:38.985888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:38.985888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:38.985888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:38.985888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:38.985888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:38.985888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.985888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:38.986264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:38.986680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:38.986680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:38.986680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.986680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:38.986973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:38.987496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:38.988040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:38.988040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:38.988040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:38.988040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:39.011751 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:39.012075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:39.012269 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:39.012483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:39.012982 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:39.013189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:39.013339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:39.013864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:39.013864 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:39.013864 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:39.013864 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:39.013864 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:39.013864 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:39.013864 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:39.013864 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:39.015084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:39.015084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:39.015084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:39.015084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:39.015084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:39.015084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:39.015084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:39.015084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:39.015084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:39.015084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:39.015084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:39.015084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:39.015675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:39.015675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:39.015675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:39.015675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:39.015675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:39.015675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:39.015675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:39.015675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:39.015675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:39.015675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:39.015675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:39.015675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:39.015675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:39.016280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:39.016280 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:39.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:39.017305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:39.017999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:39.017999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:39.017999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:41.251461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:41.252346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:41.252346 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:41.253606 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:41.285776 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:41.286224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:41.287064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:41.287064 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:41.287064 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:41.287064 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:41.287064 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:41.287064 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:41.287064 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:41.287064 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:41.287806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:41.287806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:41.287806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:41.287806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:41.287806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:41.287806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:41.287806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:41.287806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:41.287806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:41.287806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:41.287806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:41.287806 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:41.288568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:41.288568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:41.288568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:41.288568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:41.288568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:41.288568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:41.288568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:41.288568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:41.288568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:41.288568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:41.288568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:41.288568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:41.288568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:41.289113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:41.289858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:41.290778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:41.290778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:41.290778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:41.290778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:41.290778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:41.290778 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:43.949672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:43.950769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:43.950769 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:43.951052 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:43.951251 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:43.951348 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.951838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:43.951838 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:43.951838 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:43.951838 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:43.951838 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:43.951838 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:43.951838 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.951838 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:43.952855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.952855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:43.952855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:43.952855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:43.952855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:43.952855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:43.952855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:43.952855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.952855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:43.952855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:43.952855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:43.952855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.953648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:43.953648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:43.953648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:43.953648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:43.953648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:43.953648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:43.953648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:43.953648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:43.953648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:43.953648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:43.953648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:43.953648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:43.953648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:43.954553 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:43.955422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:43.956597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:43.956597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:43.956597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:43.956597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:43.956597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.956597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:43.973238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:43.974766 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:43.974766 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:43.974766 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:05:43.974766 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:43.974766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.975545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:43.975545 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:43.975545 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:43.975545 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:43.975545 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:43.975545 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:43.975545 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.975545 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:43.976779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:43.977323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:43.977323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:43.977323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:43.977323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:43.977323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:43.977323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:43.977323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:43.977323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.977323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:43.978141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.978141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:43.978141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:43.978141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:43.978141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:43.978141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:43.978141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:43.978141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:05:43.978141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:43.978141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:43.978141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.978141 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:43.978527 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:43.978872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:43.978872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:43.978872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.978872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:43.980039 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:43.980555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:43.981099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:43.981099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:43.981099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:43.981099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:43.981099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:43.981099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.981099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:43.982925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:43.983287 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:43.983402 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:43.983556 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:43.983992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:43.984238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.984732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:43.984732 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:43.984732 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:43.984732 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:43.984732 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:43.984732 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:43.984732 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.984732 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:43.985690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:43.986288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:43.986288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:43.986288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:43.986288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:43.986288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:43.986288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:43.986288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:43.986288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.986288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:43.986622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.986622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:43.986622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:43.986622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:43.986622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:43.986622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:43.986622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:43.986622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:43.986622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:43.986622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.986622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:43.986986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:43.987519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:43.987519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:43.987519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.987519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:43.987777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:43.988282 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:43.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:43.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:43.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:43.988852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:44.012144 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:44.012697 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:44.012948 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:44.013042 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:44.013394 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:44.013616 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:44.013752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:44.014896 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:44.014896 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:44.014896 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:44.014896 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:44.014896 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:44.014896 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:44.014896 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:44.014896 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:44.015276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:44.015276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:44.015276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:44.015276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:44.015276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:44.015276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:44.015276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:44.015276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:44.015276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:44.015276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:44.015276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:44.015276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:44.015835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:44.015835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:44.015835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:44.015835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:44.015835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:44.015835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:44.015835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:44.015835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:44.015835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:44.015835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:44.015835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:44.015835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:44.015835 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:44.016393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:44.017055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:44.018590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:44.018590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:44.018590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:44.018590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:44.018590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:44.018590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:45.773941 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:45.774740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:45.774740 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:45.785822 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 07 16:05:45.785994 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=105503) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 07 16:05:45.786113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 16:05:45.826411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, resource={'id': '8b456c85-5e2d-47f8-9886-ebf65932cf32', 'name': 'private', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4a00c537-ed4f-405c-b241-82e7bc2cf3fa', '9fc89727-343d-403a-acd7-db6b4950e7b1'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:25:31Z', 'updated_at': '2026-05-07T15:25:34Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 42799}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:45.826862 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8b456c85-5e2d-47f8-9886-ebf65932cf32, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:45.827398 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-b0f8152d-626b-440e-9383-919ec905ab38, external_ids_update=None, columns={'tag': 42799}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:45.827616 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.828536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:45.828536 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:45.828536 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:45.828536 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:45.828536 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:45.828536 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:45.828536 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.828536 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:45.829813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 8b456c85-5e2d-47f8-9886-ebf65932cf32 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.829813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:45.829813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 16:05:45.829813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 16:05:45.829813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 16:05:45.829813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 16:05:45.829813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 16:05:45.829813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:45.829813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:45.829813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:45.829813 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:45.830355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:45.830947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:45.830947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.830947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:45.830947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 16:05:45.867470 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, resource={'id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'name': 'public', 'tenant_id': 'adf95c4975754c649abdec2450998597', 'project_id': 'adf95c4975754c649abdec2450998597', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['6aa26064-0a62-428b-a8ed-fef52fc73984', 'adf79775-4dfa-4110-bca0-59be9560299c'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-07T15:25:37Z', 'updated_at': '2026-05-07T15:25:45Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:45.867821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:45.868488 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-2b8b3cbc-130c-4894-94e1-ab0729ec7077, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:45.868838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.869688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:45.869688 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:45.869688 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:45.869688 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:45.869688 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:45.869688 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:45.869688 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.869688 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:45.871010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 915d2372-02bc-4f18-b53f-2ab9409a0c73 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.871010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:45.871010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 16:05:45.871010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 16:05:45.871010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 16:05:45.871010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 16:05:45.871010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 16:05:45.871010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:45.871010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:45.871010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:45.871010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:45.871367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:45.871760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:45.871760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.871760 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:45.871760 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 16:05:45.899085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, resource={'id': '71aea053-d575-4d13-afc5-e6241ad6a15f', 'name': 'admin_net', 'tenant_id': '74d9c701d5cb4de6a15233057b01e413', 'project_id': '74d9c701d5cb4de6a15233057b01e413', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8e8a2cf7-97a7-4d62-ae4a-fa6c2139e811'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:30:51Z', 'updated_at': '2026-05-07T15:30:52Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 51150}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:45.899468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-71aea053-d575-4d13-afc5-e6241ad6a15f, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:45.899964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d5042c49-4c90-465a-b356-c18cf966a58b, external_ids_update=None, columns={'tag': 51150}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:45.900184 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.900984 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:45.900984 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:45.900984 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:45.900984 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:45.900984 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:45.900984 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:45.900984 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.900984 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:45.902345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 71aea053-d575-4d13-afc5-e6241ad6a15f (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.902345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:45.902345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 16:05:45.902345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 16:05:45.902345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 16:05:45.902345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 16:05:45.902345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 16:05:45.902345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:45.902345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:45.902345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:45.902345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:45.902824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:45.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:45.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:45.903422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 16:05:45.927959 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, resource={'id': '4fb571ad-ed2f-4edf-a199-cb52ff142698', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'tenant_id': '91e1e016e0f8434890282eb48ed58760', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['2e6720c7-3159-41b9-80a2-fa6cf6a18808'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-07T15:31:26Z', 'updated_at': '2026-05-07T15:31:27Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 29129}, resource_type=networks, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:45.928388 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-4fb571ad-ed2f-4edf-a199-cb52ff142698, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:45.929102 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-13b8a284-1038-44ba-86e4-667f771dfae2, external_ids_update=None, columns={'tag': 29129}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:45.929378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.930128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:45.930128 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:45.930128 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:45.930128 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:45.930128 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:45.930128 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:45.930128 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.930128 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:45.931048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 4fb571ad-ed2f-4edf-a199-cb52ff142698 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.931048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:45.931048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 16:05:45.931048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 16:05:45.931048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 16:05:45.931048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 16:05:45.931048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 07 16:05:45.931048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:45.931048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:45.931048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:45.931048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:45.931585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:45.932238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:45.932238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:45.932238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:45.932238 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 16:05:46.010449 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, resource={'id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'project_id': '91e1e016e0f8434890282eb48ed58760', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'enable_snat': True}, 'gw_port_id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-jseci', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'external_fixed_ips': [{'ip_address': '172.24.5.213', 'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984'}, {'ip_address': '2001:db8::15a', 'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 3, 'tenant_id': '91e1e016e0f8434890282eb48ed58760'}, resource_type=routers, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:46.011023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-jseci', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:46.011413 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:46.011637 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "b30ffafe-8df3-4064-bcbf-e3d1daad619c"')], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:46.011830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.012902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:46.012902 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:46.012902 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:46.012902 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:46.012902 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:46.012902 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:46.012902 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.012902 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:46.013101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Unable to update router 0938bfb8-1052-49e2-bd69-e5fda45e8a2f. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.014789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource 0938bfb8-1052-49e2-bd69-e5fda45e8a2f (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.014789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:46.014789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 16:05:46.014789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 16:05:46.014789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 16:05:46.014789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 16:05:46.014789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 07 16:05:46.014789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 07 16:05:46.014789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 07 16:05:46.014789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 07 16:05:46.014789 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:46.015103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:46.015465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:46.015465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:46.015465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:46.015465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:46.015465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:46.015465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:46.015465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:46.015465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.015465 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:46.015465 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Fixing resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports) at create/update {{(pid=105503) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 07 16:05:46.124704 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, resource={'id': 'b30ffafe-8df3-4064-bcbf-e3d1daad619c', 'name': '', 'network_id': '915d2372-02bc-4f18-b53f-2ab9409a0c73', 'project_id': '', 'mac_address': 'fa:16:3e:f6:db:65', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0938bfb8-1052-49e2-bd69-e5fda45e8a2f', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '6aa26064-0a62-428b-a8ed-fef52fc73984', 'ip_address': '172.24.5.213'}, {'subnet_id': 'adf79775-4dfa-4110-bca0-59be9560299c', 'ip_address': '2001:db8::15a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000172501', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-07T15:31:28Z', 'updated_at': '2026-05-07T15:31:31Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:46.125047 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '6aa26064-0a62-428b-a8ed-fef52fc73984 adf79775-4dfa-4110-bca0-59be9560299c', 'neutron:network_name': 'neutron-915d2372-02bc-4f18-b53f-2ab9409a0c73', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.213/24', '2001:db8::15a/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:46.125444 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=b30ffafe-8df3-4064-bcbf-e3d1daad619c, lrouter_port=lrp-b30ffafe-8df3-4064-bcbf-e3d1daad619c, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:46.125686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.126368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:46.126368 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:46.126368 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:46.126368 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:46.126368 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:46.126368 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:46.126368 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.126368 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:46.128038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Maintenance task: Failed to fix resource b30ffafe-8df3-4064-bcbf-e3d1daad619c (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.128038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:46.128038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 07 16:05:46.128038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 07 16:05:46.128038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 07 16:05:46.128038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 07 16:05:46.128038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 07 16:05:46.128038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:46.128038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:46.128038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:46.128038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:46.128642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:46.129262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:46.129262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.129262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:46.129262 np0000172501 neutron-ovn-maintenance-worker[105503]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_for_inconsistencies finished in 0.353 seconds
May 07 16:05:46.252303 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:46.252761 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:46.252884 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:46.253781 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:46.290252 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:46.290514 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.291263 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:46.291263 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:46.291263 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:46.291263 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:46.291263 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:46.291263 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:46.291263 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.291263 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:46.292159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.292159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:46.292159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:46.292159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:46.292159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:46.292159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:46.292159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:46.292159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:46.292159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:46.292159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:46.292159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:46.292159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:46.293021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:46.293021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:46.293021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:46.293021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:46.293021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:46.293021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:46.293021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:46.293021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:46.293021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:46.293021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:46.293021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:46.293021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:46.293021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:46.293705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:46.294577 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:46.295586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:46.295586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:46.295586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:46.295586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:46.295586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:46.295586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:48.950269 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:48.951294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:48.951294 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:48.952140 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:48.952530 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:48.952771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.953696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:48.953696 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:48.953696 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:48.953696 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:48.953696 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:48.953696 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:48.953696 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.953696 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:48.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:48.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:48.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:48.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:48.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:48.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:48.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:48.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:48.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:48.956075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.956761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:48.956761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:48.956761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:48.956761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:48.956761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:48.956761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:48.956761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:48.956761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:48.956761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:48.956761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:48.956761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:48.956761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:48.956761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:48.957182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.957182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:48.957859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:48.958345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:48.958852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:48.958852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.958852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:48.973964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:48.974421 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:48.974635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:48.974822 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:05:48.975393 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:48.975688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.976608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:48.976608 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:48.976608 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:48.976608 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:48.976608 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:48.976608 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:48.976608 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.976608 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:48.978248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:48.978911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:48.978911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:48.978911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:48.978911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:48.978911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:48.978911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:48.978911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:48.978911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.978911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:48.980107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.980107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:48.980107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:48.980107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:48.980107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:48.980107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:48.980107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:48.980107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:05:48.980107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:48.980107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:48.980107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.980107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:48.980728 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:48.981351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:48.981351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:48.981351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.981351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:48.982224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:48.982920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:48.983431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:48.983431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:48.983431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:48.983431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:48.983431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:48.983431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.983431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:48.983431 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:48.983749 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:48.983901 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:48.984072 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:48.984673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:48.985089 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.985901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:48.985901 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:48.985901 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:48.985901 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:48.985901 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:48.985901 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:48.985901 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.985901 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:48.986480 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:48.986916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:48.986916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:48.986916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:48.986916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:48.986916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:48.986916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:48.986916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:48.986916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.986916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:48.987182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.987182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:48.987182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:48.987182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:48.987182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:48.987182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:48.987182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:48.987182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:48.987182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:48.987182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.987182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:48.987531 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:48.988031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:48.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:48.989053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:48.989053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:48.989053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:48.989053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:48.989053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:48.989053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:48.989053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:48.989053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:48.989053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:48.989053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:48.989053 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:49.012713 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:49.012913 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:49.012998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:49.013132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:49.013634 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:49.013979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:49.014179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:49.014858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:49.014858 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:49.014858 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:49.014858 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:49.014858 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:49.014858 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:49.014858 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:49.014858 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:49.015483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:49.015483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:49.015483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:49.015483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:49.015483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:49.015483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:49.015483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:49.015483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:49.015483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:49.015483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:49.015483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:49.015483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:49.016083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:49.016083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:49.016083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:49.016083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:49.016083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:49.016083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:49.016083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:49.016083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:49.016083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:49.016083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:49.016083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:49.016083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:49.016083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:49.016688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:49.017367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:49.018131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:49.018131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:49.018131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:49.018131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:49.018131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:49.018131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:51.252829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:51.253746 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:51.253746 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:51.254698 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:51.286863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:51.287316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:51.288151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:51.288151 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:51.288151 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:51.288151 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:51.288151 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:51.288151 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:51.288151 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:51.288151 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:51.289021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:51.289021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:51.289021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:51.289021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:51.289021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:51.289021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:51.289021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:51.289021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:51.289021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:51.289021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:51.289021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:51.289021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:51.289487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:51.289487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:51.289487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:51.289487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:51.289487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:51.289487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:51.289487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:51.289487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:51.289487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:51.289487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:51.289487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:51.289487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:51.289487 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:51.289965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:51.289965 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:51.290075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:51.290522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:51.290934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:51.290934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:51.290934 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:53.950979 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:53.951804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:53.951804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:53.952585 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:53.952929 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:53.953239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.953986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:53.953986 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:53.953986 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:53.953986 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:53.953986 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:53.953986 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:53.953986 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.953986 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:53.955071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.955071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:53.955071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:53.955071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:53.955071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:53.955071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:53.955071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:53.955071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.955071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:53.955071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:53.955071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:53.955071 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.955836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:53.955836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:53.955836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:53.955836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:53.955836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:53.955836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:53.955836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:53.955836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:53.955836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:53.955836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:53.955836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:53.955836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:53.955836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:53.956177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:53.956617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:53.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:53.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:53.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:53.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:53.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:53.974540 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:53.975027 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:53.975218 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:53.975443 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:05:53.976337 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:53.976621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.980239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:53.980808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.981305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:53.981305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:53.981305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:53.981305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:53.981305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:53.981305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:53.981305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:05:53.981305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:53.981305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:53.981305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.981305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:53.981305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:53.981305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:53.981707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:53.982130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.982130 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:53.983017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:53.984045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:53.985142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:53.985142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:53.985142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:53.985142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:53.985142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:53.985142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.985142 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:53.985142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:53.985142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:53.985142 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:53.985793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:53.985793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:53.985793 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.986173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:53.986173 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:53.986173 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:53.986173 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:53.986173 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:53.986173 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:53.986173 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.986173 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:53.987045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:53.987723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:53.987723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:53.987723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:53.987723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:53.987723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:53.987723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:53.987723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:53.987723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.987723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:53.987966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.987966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:53.987966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:53.987966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:53.987966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:53.987966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:53.987966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:53.987966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:53.987966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:53.987966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.987966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:53.988299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:53.988665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:53.988665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:53.988665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.988665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:53.989188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:53.990776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:53.991787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:53.991787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:53.991787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:53.991787 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:54.013192 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:54.013744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:54.013949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:54.014157 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:54.014958 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:54.015460 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:54.015689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:54.016549 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:54.016549 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:54.016549 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:54.016549 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:54.016549 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:54.016549 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:54.016549 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:54.016549 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:54.017937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:54.017937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:54.017937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:54.017937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:54.017937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:54.017937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:54.017937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:54.017937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:54.017937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:54.017937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:54.017937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:54.017937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:54.018528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:54.018528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:54.018528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:54.018528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:54.018528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:54.018528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:54.018528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:54.018528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:54.018528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:54.018528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:54.018528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:54.018528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:54.018528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:54.019503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:54.020175 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:54.020972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:54.020972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:54.020972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:54.020972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:54.020972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:54.020972 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:56.253475 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:56.254246 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:56.254246 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:56.255464 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:56.290079 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:56.290470 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:56.291238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:56.291238 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:56.291238 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:56.291238 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:56.291238 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:56.291238 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:56.291238 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:56.291238 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:56.291991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:56.291991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:56.291991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:56.291991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:56.291991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:56.291991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:56.291991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:56.291991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:56.291991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:56.291991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:56.291991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:56.291991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:56.292468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:56.292468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:56.292468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:56.292468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:56.292468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:56.292468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:56.292468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:56.292468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:56.292468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:56.292468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:56.292468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:56.292468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:56.292468 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:56.292906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:56.293415 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:56.293997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:56.293997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:56.293997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:56.293997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:56.293997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:56.293997 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:58.951573 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:58.952069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:58.952168 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:58.953013 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:58.953411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:58.953654 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.954532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:58.954532 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:58.954532 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:58.954532 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:58.954532 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:58.954532 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:58.954532 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.954532 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:58.955664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.955664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:58.955664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:58.955664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:58.955664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:58.955664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:58.955664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:58.955664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.955664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:58.955664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:58.955664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:58.955664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.955920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:58.955920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:58.955920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:58.955920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:58.955920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:58.955920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:58.955920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:58.955920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:58.955920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:58.955920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:58.955920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:58.955920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:58.955920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:58.956172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.956172 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:58.956709 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:58.957044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:58.957438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:58.957438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.957438 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:58.974961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:58.975508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:58.975719 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:58.975916 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:05:58.976372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:58.976584 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.977195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:58.977195 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:58.977195 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:58.977195 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:58.977195 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:58.977195 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:58.977195 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.977195 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:58.978508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:58.978836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:58.978836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:58.978836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:58.978836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:58.978836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:58.978836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:58.978836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:58.978836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.978836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:58.979925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.979925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:58.979925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:58.979925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:58.979925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:58.979925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:58.979925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:58.979925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:05:58.979925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:58.979925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:58.979925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.979925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:58.980279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:58.980532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:58.980532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:58.980532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.980532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:58.982429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:58.982763 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:58.983131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:58.983131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:58.983131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:58.983131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:58.983131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:58.983131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.983131 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:58.984260 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:58.984553 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:58.984667 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:58.984794 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:58.985379 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:58.985766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.986516 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:58.986516 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:58.986516 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:58.986516 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:58.986516 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:58.986516 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:58.986516 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.986516 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:05:58.987406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:05:58.987722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:58.987722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:05:58.987722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:58.987722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:05:58.987722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:05:58.987722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:58.987722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:05:58.987722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.987722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:05:58.988339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.988339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:58.988339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:58.988339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:58.988339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:58.988339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:58.988339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:58.988339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:58.988339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:58.988339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.988339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:58.988583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:58.988846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:58.988846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:58.988846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.988846 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:58.989444 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:58.989779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:58.990115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:58.990115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:58.990115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:58.990115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:05:59.013742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:05:59.014123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:05:59.014358 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:05:59.014616 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:05:59.015113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:59.015595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:05:59.015853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:59.016673 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:05:59.016673 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:59.016673 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:05:59.016673 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:05:59.016673 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:59.016673 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:05:59.016673 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:59.016673 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:05:59.017946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:59.017946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:05:59.017946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:59.017946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:05:59.017946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:05:59.017946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:59.017946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:59.017946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:59.017946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:59.017946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:59.017946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:05:59.017946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:59.018326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:05:59.018326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:59.018326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:05:59.018326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:59.018326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:05:59.018326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:05:59.018326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:59.018326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:05:59.018326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:59.018326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:05:59.018326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:05:59.018326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:59.018326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:05:59.018697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:59.018697 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:05:59.019626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:05:59.020107 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:05:59.020638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:05:59.020638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:05:59.020638 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:01.254108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:01.254575 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:01.254720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:01.255571 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:01.289984 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:01.290231 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:01.290863 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:01.290863 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:01.290863 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:01.290863 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:01.290863 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:01.290863 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:01.290863 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:01.290863 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:01.291592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:01.291592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:01.291592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:01.291592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:01.291592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:01.291592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:01.291592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:01.291592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:01.291592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:01.291592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:01.291592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:01.291592 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:01.292150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:01.292150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:01.292150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:01.292150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:01.292150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:01.292150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:01.292150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:01.292150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:01.292150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:01.292150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:01.292150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:01.292150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:01.292150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:01.292741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:01.293359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:01.294281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:01.294281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:01.294281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:01.294281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:01.294281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:01.294281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:03.952249 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:03.953069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:03.953069 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:03.954105 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:03.954462 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:03.954700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.955568 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:03.955568 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:03.955568 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:03.955568 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:03.955568 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:03.955568 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:03.955568 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.955568 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:03.957062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.957062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:03.957062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:03.957062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:03.957062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:03.957062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:03.957062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:03.957062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.957062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:03.957062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:03.957062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:03.957062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.957962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:03.957962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:03.957962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:03.957962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:03.957962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:03.957962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:03.957962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:03.957962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:03.957962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:03.957962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:03.957962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:03.957962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:03.957962 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:03.958824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:03.959988 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:03.960876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:03.960876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:03.960876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:03.960876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:03.960876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.960876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:03.975482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:03.975899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:03.976078 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:03.976312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:06:03.976777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:03.977048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.977755 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:03.977755 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:03.977755 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:03.977755 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:03.977755 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:03.977755 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:03.977755 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.977755 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:03.978894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:03.979888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:03.979888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:03.979888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:03.979888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:03.979888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:03.979888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:03.979888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:03.979888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.979888 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:03.980490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.980490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:03.980490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:03.980490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:03.980490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:03.980490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:03.980490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:03.980490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:06:03.980490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:03.980490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:03.980490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.980490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:03.981254 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:03.982079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:03.982079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:03.982079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.982079 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:03.982377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:03.983398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:03.984436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:03.984436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:03.984436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:03.984436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:03.984436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:03.984436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.984436 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:03.984719 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:03.984983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:03.985104 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:03.985246 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:03.985670 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:03.985999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.986668 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:03.986668 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:03.986668 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:03.986668 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:03.986668 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:03.986668 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:03.986668 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.986668 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:03.987434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:03.987864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:03.987864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:03.987864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:03.987864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:03.987864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:03.987864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:03.987864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:03.987864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.987864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:03.988343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.988343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:03.988343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:03.988343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:03.988343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:03.988343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:03.988343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:03.988343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:03.988343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:03.988343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.988343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:03.988674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:03.989042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:03.989042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:03.989042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.989042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:03.989547 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:03.990010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:03.990533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:03.990533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:03.990533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:03.990533 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:04.014243 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:04.014651 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:04.014823 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:04.015058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:04.015561 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:04.015931 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:04.016163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:04.016978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:04.016978 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:04.016978 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:04.016978 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:04.016978 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:04.016978 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:04.016978 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:04.016978 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:04.018358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:04.018358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:04.018358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:04.018358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:04.018358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:04.018358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:04.018358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:04.018358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:04.018358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:04.018358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:04.018358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:04.018358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:04.019113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:04.019113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:04.019113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:04.019113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:04.019113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:04.019113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:04.019113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:04.019113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:04.019113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:04.019113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:04.019113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:04.019113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:04.019113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:04.019979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:04.019979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:04.020126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:04.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:04.021566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:04.021566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:04.021566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:06.254896 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:06.255863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:06.255863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:06.256554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:06.289639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:06.290011 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:06.290870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:06.290870 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:06.290870 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:06.290870 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:06.290870 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:06.290870 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:06.290870 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:06.290870 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:06.291723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:06.291723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:06.291723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:06.291723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:06.291723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:06.291723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:06.291723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:06.291723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:06.291723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:06.291723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:06.291723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:06.291723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:06.292452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:06.292452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:06.292452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:06.292452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:06.292452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:06.292452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:06.292452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:06.292452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:06.292452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:06.292452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:06.292452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:06.292452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:06.292452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:06.293127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:06.294196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:06.295287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:06.295287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:06.295287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:06.295287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:06.295287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:06.295287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:08.952956 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:08.953766 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:08.953839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:08.955176 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:08.955665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:08.956024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.957078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:08.957078 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:08.957078 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:08.957078 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:08.957078 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:08.957078 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:08.957078 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.957078 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:08.958720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.958720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:08.958720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:08.958720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:08.958720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:08.958720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:08.958720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:08.958720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.958720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:08.958720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:08.958720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:08.958720 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.959361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:08.959361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:08.959361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:08.959361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:08.959361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:08.959361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:08.959361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:08.959361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:08.959361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:08.959361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:08.959361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:08.959361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:08.959361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:08.959983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.959983 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:08.960622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:08.961891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:08.962862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:08.962862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.962862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:08.976031 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:08.976374 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:08.976502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:08.976587 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:06:08.977190 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:08.977642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.978496 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:08.978496 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:08.978496 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:08.978496 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:08.978496 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:08.978496 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:08.978496 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.978496 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:08.979315 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:08.980295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:08.980814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:08.980814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:08.980814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:06:08.980814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:08.980814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:08.980814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.980814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:08.980814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:08.980814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:08.980814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.980814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:08.980814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:08.980814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:08.981225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:08.981225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:08.981225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:08.981225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:08.981225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:08.981225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:08.981225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:08.981225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:08.981225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:08.981225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:08.981225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.981225 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:08.981605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:08.982041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:08.982902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:08.982902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:08.982902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:08.982902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:08.982902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:08.982902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.982902 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:08.985076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:08.985353 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:08.985461 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:08.985602 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:08.985999 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:08.986316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.986923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:08.986923 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:08.986923 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:08.986923 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:08.986923 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:08.986923 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:08.986923 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.986923 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:08.987855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:08.988445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:08.988445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:08.988445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:08.988445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:08.988445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:08.988445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:08.988445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:08.988445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.988445 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:08.989150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.989150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:08.989150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:08.989150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:08.989150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:08.989150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:08.989150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:08.989150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:08.989150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:08.989150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.989150 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:08.989667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:08.990189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:08.990189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:08.990189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.990189 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:08.990727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:08.991264 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:08.991946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:08.991946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:08.991946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:08.991946 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:09.014682 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:09.015018 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:09.015223 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:09.015470 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:09.016300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:09.016991 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:09.017269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:09.018159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:09.018159 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:09.018159 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:09.018159 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:09.018159 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:09.018159 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:09.018159 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:09.018159 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:09.019190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:09.019190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:09.019190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:09.019190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:09.019190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:09.019190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:09.019190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:09.019190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:09.019190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:09.019190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:09.019190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:09.019190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:09.019750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:09.019750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:09.019750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:09.019750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:09.019750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:09.019750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:09.019750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:09.019750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:09.019750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:09.019750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:09.019750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:09.019750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:09.019750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:09.020284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:09.020284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:09.020637 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:09.021093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:09.021640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:09.021640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:09.021640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:11.255578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:11.257899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:11.257899 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:11.258871 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:11.270198 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=434 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 16:06:11.291520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:11.291858 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:11.292606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:11.292606 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:11.292606 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:11.292606 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:11.292606 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:11.292606 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:11.292606 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:11.292606 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:11.293484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:11.293484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:11.293484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:11.293484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:11.293484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:11.293484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:11.293484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:11.293484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:11.293484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:11.293484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:11.293484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:11.293484 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:11.294016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:11.294016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:11.294016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:11.294016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:11.294016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:11.294016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:11.294016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:11.294016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:11.294016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:11.294016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:11.294016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:11.294016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:11.294016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:11.294625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:11.294625 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:11.295064 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:11.297001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:11.297737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:11.297737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:11.297737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:13.953614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:13.955233 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:13.955233 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:13.955233 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:13.955233 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:13.955562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.956407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:13.956407 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:13.956407 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:13.956407 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:13.956407 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:13.956407 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:13.956407 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.956407 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:13.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:13.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:13.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:13.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:13.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:13.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:13.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:13.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:13.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:13.957118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.957404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:13.957404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:13.957404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:13.957404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:13.957404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:13.957404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:13.957404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:13.957404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:13.957404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:13.957404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:13.957404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:13.957404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:13.957404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:13.957645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.957645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:13.957854 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:13.958192 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:13.958545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:13.958545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.958545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:13.976480 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:13.976845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:13.977060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:13.977159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:06:13.977847 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:13.978121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.978931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:13.978931 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:13.978931 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:13.978931 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:13.978931 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:13.978931 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:13.978931 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.978931 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:13.980069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:13.980950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:13.980950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:13.980950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:13.980950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:13.980950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:13.980950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:13.980950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:13.980950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.980950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:13.981608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.981608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:13.981608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:13.981608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:13.981608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:13.981608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:13.981608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:13.981608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:06:13.981608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:13.981608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:13.981608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.981608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:13.981973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:13.982325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:13.982325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:13.982325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.982325 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:13.983899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:13.984495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:13.984980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:13.984980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:13.984980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:13.984980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:13.984980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:13.984980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.984980 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:13.985500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:13.985839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:13.986010 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:13.986231 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:13.986653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:13.986901 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.987627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:13.987627 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:13.987627 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:13.987627 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:13.987627 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:13.987627 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:13.987627 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.987627 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:13.988467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:13.988872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:13.988872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:13.988872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:13.988872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:13.988872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:13.988872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:13.988872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:13.988872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.988872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:13.989358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.989358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:13.989358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:13.989358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:13.989358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:13.989358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:13.989358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:13.989358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:13.989358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:13.989358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.989358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:13.990385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:13.990740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:13.991314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:13.991832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:13.991832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:13.991832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:13.991832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:13.991832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:13.991832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:13.991832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:13.991832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:13.991832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:13.991832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:13.991832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:14.015117 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:14.015568 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:14.015702 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:14.015927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:14.016552 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:14.017006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:14.017118 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:14.017786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:14.017786 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:14.017786 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:14.017786 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:14.017786 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:14.017786 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:14.017786 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:14.017786 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:14.019057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:14.019057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:14.019057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:14.019057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:14.019057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:14.019057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:14.019057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:14.019057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:14.019057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:14.019057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:14.019057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:14.019057 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:14.019735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:14.019735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:14.019735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:14.019735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:14.019735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:14.019735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:14.019735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:14.019735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:14.019735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:14.019735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:14.019735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:14.019735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:14.019735 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:14.020066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:14.020066 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:14.020447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:14.021063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:14.021828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:14.021828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:14.021828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:16.256422 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:16.257038 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:16.257123 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:16.258404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:16.295610 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:16.295990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:16.296841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:16.296841 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:16.296841 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:16.296841 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:16.296841 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:16.296841 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:16.296841 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:16.296841 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:16.297947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:16.297947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:16.297947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:16.297947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:16.297947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:16.297947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:16.297947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:16.297947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:16.297947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:16.297947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:16.297947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:16.297947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:16.298612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:16.298612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:16.298612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:16.298612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:16.298612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:16.298612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:16.298612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:16.298612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:16.298612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:16.298612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:16.298612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:16.298612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:16.298612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:16.299149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:16.299848 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:16.300640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:16.300640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:16.300640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:16.300640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:16.300640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:16.300640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:18.954284 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:18.955118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:18.955118 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:18.955969 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:18.956145 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:18.956308 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.956866 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:18.956866 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:18.956866 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:18.956866 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:18.956866 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:18.956866 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:18.956866 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.956866 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:18.958627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.958627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:18.958627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:18.958627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:18.958627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:18.958627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:18.958627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:18.958627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.958627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:18.958627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:18.958627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:18.958627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.959072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:18.959072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:18.959072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:18.959072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:18.959072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:18.959072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:18.959072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:18.959072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:18.959072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:18.959072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:18.959072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:18.959072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:18.959072 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:18.959431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.959431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:18.960358 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:18.961324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:18.962161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:18.962161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.962161 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:18.976961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:18.977478 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:18.977654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:18.977839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:06:18.978159 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:18.978302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.978855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:18.978855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:18.978855 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:18.978855 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:18.978855 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:18.978855 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:18.978855 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.978855 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:18.980054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:18.980882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:18.980882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:18.980882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:18.980882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:18.980882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:18.980882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:18.980882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:18.980882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.980882 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:18.981599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.981599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:18.981599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:18.981599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:18.981599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:18.981599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:18.981599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:18.981599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:06:18.981599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:18.981599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:18.981599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.981599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:18.983119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:18.983537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:18.984006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:18.984655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:18.984655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:18.984655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:18.984655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:18.984655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:18.984655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:18.984655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:18.984655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:18.984655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:18.984655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:18.984655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.984655 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:18.985971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:18.986343 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:18.986523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:18.986712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:18.986992 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:18.987145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.987601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:18.987601 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:18.987601 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:18.987601 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:18.987601 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:18.987601 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:18.987601 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.987601 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:18.988458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:18.988925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:18.988925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:18.988925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:18.988925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:18.988925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:18.988925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:18.988925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:18.988925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.988925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:18.989341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.989341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:18.989341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:18.989341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:18.989341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:18.989341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:18.989341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:18.989341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:18.989341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:18.989341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.989341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:18.989775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:18.990188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:18.990188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:18.990188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.990188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:18.990589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:18.991191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:18.991749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:18.991749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:18.991749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:18.991749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:19.015647 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:19.016132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:19.016332 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:19.016583 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:19.016861 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:19.017014 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:19.017137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:19.017586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:19.017586 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:19.017586 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:19.017586 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:19.017586 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:19.017586 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:19.017586 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:19.017586 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:19.019013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:19.019013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:19.019013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:19.019013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:19.019013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:19.019013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:19.019013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:19.019013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:19.019013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:19.019013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:19.019013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:19.019013 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:19.019442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:19.019442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:19.019442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:19.019442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:19.019442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:19.019442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:19.019442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:19.019442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:19.019442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:19.019442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:19.019442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:19.019442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:19.019442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:19.019772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:19.019772 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:19.020502 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:19.020992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:19.021522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:19.021522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:19.021522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:21.256894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:21.257677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:21.257677 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:21.259130 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:21.296257 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:21.296689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:21.297630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:21.297630 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:21.297630 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:21.297630 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:21.297630 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:21.297630 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:21.297630 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:21.297630 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:21.298756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:21.298756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:21.298756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:21.298756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:21.298756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:21.298756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:21.298756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:21.298756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:21.298756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:21.298756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:21.298756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:21.298756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:21.299134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:21.299134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:21.299134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:21.299134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:21.299134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:21.299134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:21.299134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:21.299134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:21.299134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:21.299134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:21.299134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:21.299134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:21.299134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:21.299500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:21.299500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:21.300133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:21.300604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:21.301099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:21.301099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:21.301099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:23.954887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:23.955909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:23.955909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:23.958164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:23.958164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:23.958164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.958541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:23.958541 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:23.958541 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:23.958541 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:23.958541 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:23.958541 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:23.958541 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.958541 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:23.960180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.960180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:23.960180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:23.960180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:23.960180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:23.960180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:23.960180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:23.960180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.960180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:23.960180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:23.960180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:23.960180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.960729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:23.960729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:23.960729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:23.960729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:23.960729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:23.960729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:23.960729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:23.960729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:23.960729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:23.960729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:23.960729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:23.960729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:23.960729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:23.961188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.961188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:23.961899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:23.962407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:23.962905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:23.962905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.962905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:23.977624 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:23.978004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:23.978113 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:23.978270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:06:23.978625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:23.978926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.979571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:23.979571 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:23.979571 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:23.979571 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:23.979571 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:23.979571 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:23.979571 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.979571 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:23.980486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:23.980895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:23.980895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:23.980895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:23.980895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:23.980895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:23.980895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:23.980895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:23.980895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.980895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:23.981376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.981376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:23.981376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:23.981376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:23.981376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:23.981376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:23.981376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:23.981376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:06:23.981376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:23.981376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:23.981376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.981376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:23.981748 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:23.982422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:23.982422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:23.982422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.982422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:23.982699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:23.983576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:23.984471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:23.984471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:23.984471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:23.984471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:23.984471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:23.984471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.984471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:23.986342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:23.986615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:23.986768 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:23.986946 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:23.987621 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:23.987621 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.987976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:23.987976 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:23.987976 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:23.987976 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:23.987976 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:23.987976 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:23.987976 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.987976 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:23.988727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:23.989404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:23.989404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:23.989404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:23.989404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:23.989404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:23.989404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:23.989404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:23.989404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.989404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:23.989781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.989781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:23.989781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:23.989781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:23.989781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:23.989781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:23.989781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:23.989781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:23.989781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:23.989781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.989781 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:23.990202 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:23.990691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:23.990691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:23.990691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.990691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:23.991532 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:23.992523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:23.993898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:23.993898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:23.993898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:23.993898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:24.016253 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:24.016680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:24.017015 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:24.017389 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:24.017960 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:24.018417 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:24.018672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:24.019525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:24.019525 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:24.019525 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:24.019525 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:24.019525 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:24.019525 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:24.019525 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:24.019525 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:24.021597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:24.021597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:24.021597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:24.021597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:24.021597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:24.021597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:24.021597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:24.021597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:24.021597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:24.021597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:24.021597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:24.021597 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:24.022007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:24.022007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:24.022007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:24.022007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:24.022007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:24.022007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:24.022007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:24.022007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:24.022007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:24.022007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:24.022007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:24.022007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:24.022007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:24.022388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:24.022388 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:24.023862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:24.024349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:24.024878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:24.024878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:24.024878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:26.257580 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:26.258661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:26.258661 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:26.258805 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:26.294225 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:26.294387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:26.295004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:26.295004 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:26.295004 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:26.295004 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:26.295004 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:26.295004 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:26.295004 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:26.295004 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:26.295958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:26.295958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:26.295958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:26.295958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:26.295958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:26.295958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:26.295958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:26.295958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:26.295958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:26.295958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:26.295958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:26.295958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:26.296678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:26.296678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:26.296678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:26.296678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:26.296678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:26.296678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:26.296678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:26.296678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:26.296678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:26.296678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:26.296678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:26.296678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:26.296678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:26.297412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:26.298274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:26.299334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:26.299334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:26.299334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:26.299334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:26.299334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:26.299334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:28.955497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:28.956342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:28.956342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:28.957376 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:28.957694 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:28.957932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.958784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:28.958784 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:28.958784 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:28.958784 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:28.958784 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:28.958784 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:28.958784 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.958784 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:28.960426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.960426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:28.960426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:28.960426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:28.960426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:28.960426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:28.960426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:28.960426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.960426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:28.960426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:28.960426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:28.960426 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.961040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:28.961040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:28.961040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:28.961040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:28.961040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:28.961040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:28.961040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:28.961040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:28.961040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:28.961040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:28.961040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:28.961040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:28.961040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:28.961701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.961701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:28.962242 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:28.963037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:28.963589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:28.963589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.963589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:28.978077 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:28.978529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:28.978685 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:28.978856 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:06:28.979360 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:28.979645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.980412 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:28.980412 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:28.980412 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:28.980412 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:28.980412 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:28.980412 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:28.980412 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.980412 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:28.985881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:28.986723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:28.986723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:28.986723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:28.986723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:28.986723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:28.986723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:28.986723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:28.986723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.986723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:28.987786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.987786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:28.987786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:28.987786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:28.987786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:28.987786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:28.987786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:28.987786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:06:28.987786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:28.987786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:28.987786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.987786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:28.988400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:28.989045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:28.989045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:28.989045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.989045 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:28.989045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:28.989624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:28.990046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:28.990565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:28.990565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:28.990565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:28.990565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:28.990565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:28.990565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.990565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:28.990565 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:28.990565 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:28.990565 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:28.990877 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:28.990877 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.991500 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:28.991500 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:28.991500 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:28.991500 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:28.991500 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:28.991500 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:28.991500 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.991500 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:28.992219 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:28.992633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:28.992633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:28.992633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:28.992633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:28.992633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:28.992633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:28.992633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:28.992633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.992633 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:28.993155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.993155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:28.993155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:28.993155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:28.993155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:28.993155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:28.993155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:28.993155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:28.993155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:28.993155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.993155 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:28.993494 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:28.993864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:28.993864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:28.993864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.993864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:28.994636 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:28.995112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:28.995635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:28.995635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:28.995635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:28.995635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:29.016810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:29.017187 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:29.017395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:29.017657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:29.018148 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:29.018529 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:29.018745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:29.019483 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:29.019483 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:29.019483 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:29.019483 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:29.019483 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:29.019483 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:29.019483 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:29.019483 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:29.020823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:29.020823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:29.020823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:29.020823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:29.020823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:29.020823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:29.020823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:29.020823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:29.020823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:29.020823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:29.020823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:29.020823 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:29.021682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:29.021682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:29.021682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:29.021682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:29.021682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:29.021682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:29.021682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:29.021682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:29.021682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:29.021682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:29.021682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:29.021682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:29.021682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:29.022511 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:29.023323 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:29.024307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:29.024307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:29.024307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:29.024307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:29.024307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:29.024307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:31.258245 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:31.258820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:31.258967 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:31.260165 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:31.296119 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:31.296605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:31.297551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:31.297551 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:31.297551 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:31.297551 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:31.297551 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:31.297551 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:31.297551 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:31.297551 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:31.298367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:31.298367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:31.298367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:31.298367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:31.298367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:31.298367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:31.298367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:31.298367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:31.298367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:31.298367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:31.298367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:31.298367 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:31.299026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:31.299026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:31.299026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:31.299026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:31.299026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:31.299026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:31.299026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:31.299026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:31.299026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:31.299026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:31.299026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:31.299026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:31.299026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:31.299644 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:31.300324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:31.301123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:31.301123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:31.301123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:31.301123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:31.301123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:31.301123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:33.955955 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:33.956574 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:33.956668 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:33.957918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:33.958452 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:33.958743 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.960015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:33.960015 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:33.960015 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:33.960015 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:33.960015 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:33.960015 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:33.960015 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.960015 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:33.961100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.961100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:33.961100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:33.961100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:33.961100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:33.961100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:33.961100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:33.961100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.961100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:33.961100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:33.961100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:33.961100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.961497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:33.961497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:33.961497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:33.961497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:33.961497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:33.961497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:33.961497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:33.961497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:33.961497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:33.961497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:33.961497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:33.961497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:33.961497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:33.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.961836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:33.962158 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:33.962618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:33.963126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:33.963126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.963126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:33.978418 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:33.978625 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:33.978723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:33.978833 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:06:33.979489 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:33.979817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.980706 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:33.980706 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:33.980706 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:33.980706 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:33.980706 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:33.980706 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:33.980706 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.980706 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:33.981853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:33.982574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:33.982574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:33.982574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:33.982574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:33.982574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:33.982574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:33.982574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:33.982574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.982574 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:33.983115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.983115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:33.983115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:33.983115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:33.983115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:33.983115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:33.983115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:33.983115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:06:33.983115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:33.983115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:33.983115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.983115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:33.983762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:33.984361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:33.984361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:33.984361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.984361 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:33.984696 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:33.985147 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:33.985719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:33.985719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:33.985719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:33.985719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:33.985719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:33.985719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.985719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:33.989873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:33.990023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:33.990097 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:33.990197 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:33.990714 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:33.991092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.992121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:33.992121 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:33.992121 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:33.992121 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:33.992121 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:33.992121 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:33.992121 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.992121 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:33.994006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:33.994478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:33.994478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:33.994478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:33.994478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:33.994478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:33.994478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:33.994478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:33.994478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.994478 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:33.996406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.996406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:33.996406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:33.996406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:33.996406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:33.996406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:33.996406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:33.996406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:33.996406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:33.996406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.996406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:33.996765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:33.997149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:33.997149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:33.997149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.997149 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:33.997940 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:33.998452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:33.998968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:33.998968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:33.998968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:33.998968 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:34.017300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:34.017547 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:34.017619 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:34.017788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:34.018350 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:34.018712 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:34.018978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:34.019851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:34.019851 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:34.019851 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:34.019851 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:34.019851 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:34.019851 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:34.019851 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:34.019851 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:34.020819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:34.020819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:34.020819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:34.020819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:34.020819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:34.020819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:34.020819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:34.020819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:34.020819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:34.020819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:34.020819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:34.020819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:34.021453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:34.021453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:34.021453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:34.021453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:34.021453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:34.021453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:34.021453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:34.021453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:34.021453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:34.021453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:34.021453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:34.021453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:34.021453 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:34.022120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:34.022929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:34.024054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:34.024054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:34.024054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:34.024054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:34.024054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:34.024054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:36.258822 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:36.260000 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:36.260196 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:36.261556 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:36.296152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:36.296692 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:36.297859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:36.297859 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:36.297859 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:36.297859 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:36.297859 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:36.297859 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:36.297859 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:36.297859 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:36.298916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:36.298916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:36.298916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:36.298916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:36.298916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:36.298916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:36.298916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:36.298916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:36.298916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:36.298916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:36.298916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:36.298916 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:36.299611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:36.299611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:36.299611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:36.299611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:36.299611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:36.299611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:36.299611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:36.299611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:36.299611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:36.299611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:36.299611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:36.299611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:36.299611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:36.300232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:36.301001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:36.301886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:36.301886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:36.301886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:36.301886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:36.301886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:36.301886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:38.956578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:38.957364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:38.957364 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:38.958636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:38.959060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:38.959450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.960434 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:38.960434 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:38.960434 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:38.960434 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:38.960434 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:38.960434 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:38.960434 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.960434 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:38.961922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.961922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:38.961922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:38.961922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:38.961922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:38.961922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:38.961922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:38.961922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.961922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:38.961922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:38.961922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:38.961922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.963009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:38.963009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:38.963009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:38.963009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:38.963009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:38.963009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:38.963009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:38.963009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:38.963009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:38.963009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:38.963009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:38.963009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:38.963009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:38.964004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:38.964876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:38.965931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:38.965931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:38.965931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:38.965931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:38.965931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.965931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:38.978783 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:38.979003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:38.979086 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:38.979224 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:06:38.979922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:38.980272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.981128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:38.981128 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:38.981128 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:38.981128 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:38.981128 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:38.981128 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:38.981128 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.981128 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:38.982333 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:38.983128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:38.983128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:38.983128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:38.983128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:38.983128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:38.983128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:38.983128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:38.983128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.983128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:38.983649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.983649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:38.983649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:38.983649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:38.983649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:38.983649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:38.983649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:38.983649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:06:38.983649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:38.983649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:38.983649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.983649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:38.984304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:38.984914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:38.984914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:38.984914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.984914 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:38.985295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:38.986151 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:38.987030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:38.987030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:38.987030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:38.987030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:38.987030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:38.987030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.987030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:38.990262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:38.990535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:38.990615 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:38.990762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:38.991582 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:38.992038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.992887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:38.992887 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:38.992887 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:38.992887 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:38.992887 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:38.992887 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:38.992887 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.992887 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:38.993607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:38.994411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:38.994411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:38.994411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:38.994411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:38.994411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:38.994411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:38.994411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:38.994411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.994411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:38.994718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.994718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:38.994718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:38.994718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:38.994718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:38.994718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:38.994718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:38.994718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:38.994718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:38.994718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.994718 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:38.995081 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:38.995620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:38.996055 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:38.996660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:38.996660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:38.996660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:38.996660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:38.996660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:38.996660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:38.996660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:38.996660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:38.996660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:38.996660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:38.996660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:39.017746 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:39.018201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:39.018488 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:39.018795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:39.019554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:39.020010 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:39.020339 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:39.021200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:39.021200 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:39.021200 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:39.021200 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:39.021200 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:39.021200 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:39.021200 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:39.021200 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:39.022827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:39.022827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:39.022827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:39.022827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:39.022827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:39.022827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:39.022827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:39.022827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:39.022827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:39.022827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:39.022827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:39.022827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:39.023414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:39.023414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:39.023414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:39.023414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:39.023414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:39.023414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:39.023414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:39.023414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:39.023414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:39.023414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:39.023414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:39.023414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:39.023414 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:39.024017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:39.024017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:39.024855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:39.025594 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:39.026477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:39.026477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:39.026477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:41.260138 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:41.260980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:41.260980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:41.261227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:41.294700 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:41.295063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:41.296136 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:41.296136 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:41.296136 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:41.296136 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:41.296136 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:41.296136 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:41.296136 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:41.296136 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:41.297729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:41.297729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:41.297729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:41.297729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:41.297729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:41.297729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:41.297729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:41.297729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:41.297729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:41.297729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:41.297729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:41.297729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:41.298307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:41.298307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:41.298307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:41.298307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:41.298307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:41.298307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:41.298307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:41.298307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:41.298307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:41.298307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:41.298307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:41.298307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:41.298307 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:41.298900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:41.298900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:41.299458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:41.300200 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:41.301035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:41.301035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:41.301035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:43.957385 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:43.958483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:43.958483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:43.959322 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:43.959673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:43.959925 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.960898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:43.960898 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:43.960898 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:43.960898 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:43.960898 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:43.960898 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:43.960898 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.960898 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:43.962244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.962244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:43.962244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:43.962244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:43.962244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:43.962244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:43.962244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:43.962244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.962244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:43.962244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:43.962244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:43.962244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.962939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:43.962939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:43.962939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:43.962939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:43.962939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:43.962939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:43.962939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:43.962939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:43.962939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:43.962939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:43.962939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:43.962939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:43.962939 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:43.963586 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:43.964366 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:43.965353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:43.965353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:43.965353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:43.965353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:43.965353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.965353 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:43.979378 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:43.979659 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:43.979736 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:43.979801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:06:43.980426 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:43.980824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.981563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:43.981563 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:43.981563 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:43.981563 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:43.981563 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:43.981563 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:43.981563 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.981563 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:43.982101 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:43.982883 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:43.983583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:43.983583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:43.983583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:06:43.983583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:43.983583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:43.983583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.983583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:43.983583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:43.983583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:43.983583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.983583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:43.983583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:43.983583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:43.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:43.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:43.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:43.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:43.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:43.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:43.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:43.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:43.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:43.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:43.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.984186 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:43.984834 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:43.985507 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:43.986018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:43.986018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:43.986018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:43.986018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:43.986018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:43.986018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.986018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:43.990657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:43.990889 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:43.990964 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:43.991074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:43.991556 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:43.991915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.992604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:43.992604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:43.992604 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:43.992604 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:43.992604 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:43.992604 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:43.992604 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.992604 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:43.993164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:43.993792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:43.994505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:43.994505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:43.994505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:43.994505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:43.994505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.994505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:43.994505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:43.994505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:43.994505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.994505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:43.994505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:43.994505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:43.994505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:43.995100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:43.995100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:43.995100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:43.995100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:43.995100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:43.995100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:43.995100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:43.995100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:43.995100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:43.995100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.995100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:43.995100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:43.995761 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:43.997061 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:44.018509 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:44.018902 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:44.019035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:44.019129 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:44.019645 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:44.020007 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:44.020276 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:44.021065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:44.021065 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:44.021065 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:44.021065 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:44.021065 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:44.021065 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:44.021065 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:44.021065 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:44.021967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:44.021967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:44.021967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:44.021967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:44.021967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:44.021967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:44.021967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:44.021967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:44.021967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:44.021967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:44.021967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:44.021967 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:44.022911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:44.022911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:44.022911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:44.022911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:44.022911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:44.022911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:44.022911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:44.022911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:44.022911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:44.022911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:44.022911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:44.022911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:44.022911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:44.023508 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:44.024347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:44.025363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:44.025363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:44.025363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:44.025363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:44.025363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:44.025363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:46.260792 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:46.261525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:46.261525 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:46.262187 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:46.296254 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:46.296537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:46.297245 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:46.297245 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:46.297245 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:46.297245 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:46.297245 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:46.297245 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:46.297245 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:46.297245 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:46.297976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:46.297976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:46.297976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:46.297976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:46.297976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:46.297976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:46.297976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:46.297976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:46.297976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:46.297976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:46.297976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:46.297976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:46.298575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:46.298575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:46.298575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:46.298575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:46.298575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:46.298575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:46.298575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:46.298575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:46.298575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:46.298575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:46.298575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:46.298575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:46.298575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:46.299114 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:46.299827 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:46.300681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:46.300681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:46.300681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:46.300681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:46.300681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:46.300681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:48.958152 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:48.958971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:48.958971 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:48.960410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:48.960710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:48.960919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.961812 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:48.961812 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:48.961812 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:48.961812 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:48.961812 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:48.961812 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:48.961812 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.961812 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:48.963575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.963575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:48.963575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:48.963575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:48.963575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:48.963575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:48.963575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:48.963575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.963575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:48.963575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:48.963575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:48.963575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.964157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:48.964157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:48.964157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:48.964157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:48.964157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:48.964157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:48.964157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:48.964157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:48.964157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:48.964157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:48.964157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:48.964157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:48.964157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:48.964731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.964731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:48.965292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:48.965885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:48.966392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:48.966392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.966392 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:48.980104 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:48.980570 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:48.980766 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:48.980960 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:06:48.981934 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:48.982239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.983094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:48.983094 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:48.983094 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:48.983094 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:48.983094 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:48.983094 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:48.983094 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.983094 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:48.984431 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:48.985239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:48.985239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:48.985239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:48.985239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:48.985239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:48.985239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:48.985239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:48.985239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.985239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:48.985819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.985819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:48.985819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:48.985819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:48.985819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:48.985819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:48.985819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:48.985819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:06:48.985819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:48.985819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:48.985819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.985819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:48.986473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:48.987119 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:48.987766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:48.988722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:48.988722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:48.988722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:48.988722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:48.988722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:48.988722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:48.988722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:48.988722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:48.988722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:48.988722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:48.988722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.988722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:48.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:48.991372 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:48.991478 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:48.991614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:48.992135 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:48.992404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.993007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:48.993007 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:48.993007 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:48.993007 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:48.993007 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:48.993007 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:48.993007 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.993007 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:48.993838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:48.994320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:48.994320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:48.994320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:48.994320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:48.994320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:48.994320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:48.994320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:48.994320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.994320 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:48.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:48.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:48.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:48.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:48.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:48.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:48.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:48.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:48.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.994693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:48.995042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:48.995471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:48.995471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:48.995471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.995471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:48.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:48.996326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:48.996871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:48.996871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:48.996871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:48.996871 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:49.019129 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:49.019528 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:49.019704 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:49.019939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:49.020652 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:49.020932 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:49.021103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:49.021926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:49.021926 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:49.021926 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:49.021926 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:49.021926 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:49.021926 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:49.021926 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:49.021926 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:49.023381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:49.023381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:49.023381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:49.023381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:49.023381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:49.023381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:49.023381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:49.023381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:49.023381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:49.023381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:49.023381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:49.023381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:49.023938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:49.023938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:49.023938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:49.023938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:49.023938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:49.023938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:49.023938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:49.023938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:49.023938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:49.023938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:49.023938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:49.023938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:49.023938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:49.024598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:49.024598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:49.024900 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:49.025700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:49.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:49.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:49.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:51.261528 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:51.262368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:51.262368 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:51.262848 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:51.294250 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:51.294656 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:51.295473 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:51.295473 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:51.295473 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:51.295473 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:51.295473 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:51.295473 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:51.295473 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:51.295473 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:51.296344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:51.296344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:51.296344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:51.296344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:51.296344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:51.296344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:51.296344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:51.296344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:51.296344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:51.296344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:51.296344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:51.296344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:51.296819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:51.296819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:51.296819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:51.296819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:51.296819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:51.296819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:51.296819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:51.296819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:51.296819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:51.296819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:51.296819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:51.296819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:51.296819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:51.297191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:51.297191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:51.297283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:51.297732 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:51.298249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:51.298249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:51.298249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:53.959039 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:53.960268 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:53.960479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:53.961715 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:53.961957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:53.962073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.962768 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:53.962768 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:53.962768 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:53.962768 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:53.962768 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:53.962768 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:53.962768 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.962768 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:53.965094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.965094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:53.965094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:53.965094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:53.965094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:53.965094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:53.965094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:53.965094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.965094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:53.965094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:53.965094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:53.965094 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.965769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:53.965769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:53.965769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:53.965769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:53.965769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:53.965769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:53.965769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:53.965769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:53.965769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:53.965769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:53.965769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:53.965769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:53.965769 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:53.966442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.966442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:53.966897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:53.967841 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:53.968548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:53.968548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.968548 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:53.980666 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:53.981534 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:53.981738 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:53.981949 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:06:53.982616 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:53.982876 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.983791 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:53.983791 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:53.983791 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:53.983791 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:53.983791 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:53.983791 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:53.983791 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.983791 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:53.985572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:53.986387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:53.986387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:53.986387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:53.986387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:53.986387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:53.986387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:53.986387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:53.986387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.986387 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:53.987541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.987541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:53.987541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:53.987541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:53.987541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:53.987541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:53.987541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:53.987541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:06:53.987541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:53.987541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:53.987541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.987541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:53.988178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:53.988861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:53.988861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:53.988861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.988861 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:53.989058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:53.989814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:53.990693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:53.990693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:53.990693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:53.990693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:53.990693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:53.990693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.990693 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:53.991455 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:53.991845 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:53.991962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:53.992096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:53.992550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:53.992819 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.993447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:53.993447 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:53.993447 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:53.993447 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:53.993447 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:53.993447 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:53.993447 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.993447 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:53.994406 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:53.994796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:53.994796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:53.994796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:53.994796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:53.994796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:53.994796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:53.994796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:53.994796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.994796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:53.995345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.995345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:53.995345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:53.995345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:53.995345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:53.995345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:53.995345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:53.995345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:53.995345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:53.995345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.995345 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:53.995700 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:53.996062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:53.996062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:53.996062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.996062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:53.996589 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:53.997058 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:53.997551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:53.997551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:53.997551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:53.997551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:54.019771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:54.020262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:54.020531 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:54.020825 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:54.021814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:54.022225 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:54.022490 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:54.023540 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:54.023540 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:54.023540 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:54.023540 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:54.023540 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:54.023540 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:54.023540 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:54.023540 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:54.025113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:54.025113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:54.025113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:54.025113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:54.025113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:54.025113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:54.025113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:54.025113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:54.025113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:54.025113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:54.025113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:54.025113 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:54.025783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:54.025783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:54.025783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:54.025783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:54.025783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:54.025783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:54.025783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:54.025783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:54.025783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:54.025783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:54.025783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:54.025783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:54.025783 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:54.026404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:54.026404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:54.026938 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:54.028073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:54.029411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:54.029411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:54.029411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:56.262107 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:56.262946 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:56.263138 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:56.264814 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:56.298864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:56.299305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:56.300216 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:56.300216 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:56.300216 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:56.300216 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:56.300216 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:56.300216 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:56.300216 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:56.300216 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:56.302859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:56.302859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:56.302859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:56.302859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:56.302859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:56.302859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:56.302859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:56.302859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:56.302859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:56.302859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:56.302859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:56.302859 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:56.303285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:56.303285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:56.303285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:56.303285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:56.303285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:56.303285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:56.303285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:56.303285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:56.303285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:56.303285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:56.303285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:56.303285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:56.303285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:56.303618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:56.303618 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:56.305018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:56.305463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:56.305926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:56.305926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:56.305926 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:58.960120 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:58.961001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:58.961001 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:58.961723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:58.962046 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:58.962321 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.963384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:58.963384 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:58.963384 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:58.963384 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:58.963384 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:58.963384 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:58.963384 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.963384 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:58.964344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.964344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:58.964344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:58.964344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:58.964344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:58.964344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:58.964344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:58.964344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.964344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:58.964344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:58.964344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:58.964344 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.964958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:58.964958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:58.964958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:58.964958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:58.964958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:58.964958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:58.964958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:58.964958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:58.964958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:58.964958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:58.964958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:58.964958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:58.964958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:58.965604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:58.966346 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:58.967104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:58.967104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:58.967104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:58.967104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:58.967104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.967104 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:58.981266 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:58.981744 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:58.981836 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:58.981957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:06:58.982482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:58.982870 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.983599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:58.983599 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:58.983599 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:58.983599 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:58.983599 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:58.983599 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:58.983599 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.983599 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:58.984417 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:58.985112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:58.985112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:58.985112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:58.985112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:58.985112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:58.985112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:58.985112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:58.985112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.985112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:58.985522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.985522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:58.985522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:58.985522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:58.985522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:58.985522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:58.985522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:58.985522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:06:58.985522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:58.985522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:58.985522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.985522 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:58.987279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:58.987905 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:58.988517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:58.989350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:58.989350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:58.989350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:58.989350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:58.989350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:58.989350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:58.989350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:58.989350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:58.989350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:58.989350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:58.989350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.989350 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:58.991893 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:58.992521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:58.992762 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:58.992983 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:58.993649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:58.994038 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.995127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:58.995127 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:58.995127 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:58.995127 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:58.995127 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:58.995127 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:58.995127 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.995127 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:06:58.996524 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:06:58.996937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:58.996937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:06:58.996937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:58.996937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:06:58.996937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:06:58.996937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:58.996937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:06:58.996937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.996937 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:06:58.998121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.998121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:58.998121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:58.998121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:58.998121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:58.998121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:58.998121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:58.998121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:58.998121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:58.998121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.998121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:58.998687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:58.999075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:58.999075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:58.999075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.999075 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:58.999831 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:59.000316 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:59.000833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:59.000833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:59.000833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:59.000833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:06:59.020395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:06:59.020858 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:06:59.021071 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:06:59.021427 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:06:59.021890 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:59.022285 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:06:59.022518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:59.023385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:06:59.023385 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:59.023385 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:06:59.023385 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:06:59.023385 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:59.023385 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:06:59.023385 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:59.023385 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:06:59.024519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:59.024519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:06:59.024519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:59.024519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:06:59.024519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:06:59.024519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:59.024519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:59.024519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:59.024519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:59.024519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:59.024519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:06:59.024519 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:59.025059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:06:59.025059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:59.025059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:06:59.025059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:59.025059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:06:59.025059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:06:59.025059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:59.025059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:06:59.025059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:59.025059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:06:59.025059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:06:59.025059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:59.025059 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:06:59.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:06:59.026065 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:06:59.027198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:06:59.027198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:06:59.027198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:06:59.027198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:06:59.027198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:06:59.027198 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:01.263004 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:01.264085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:01.264085 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:01.265240 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:01.303774 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:01.304243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:01.305472 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:01.305472 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:01.305472 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:01.305472 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:01.305472 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:01.305472 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:01.305472 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:01.305472 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:01.306745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:01.306745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:01.306745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:01.306745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:01.306745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:01.306745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:01.306745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:01.306745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:01.306745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:01.306745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:01.306745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:01.306745 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:01.307542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:01.307542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:01.307542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:01.307542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:01.307542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:01.307542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:01.307542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:01.307542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:01.307542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:01.307542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:01.307542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:01.307542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:01.307542 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:01.308248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:01.308776 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:01.309432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:01.309432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:01.309432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:01.309432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:01.309432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:01.309432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:03.960806 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:03.961551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:03.962008 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:03.962937 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:03.963232 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:03.963482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.964357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:03.964357 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:03.964357 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:03.964357 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:03.964357 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:03.964357 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:03.964357 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.964357 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:03.965729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.965729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:03.965729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:03.965729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:03.965729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:03.965729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:03.965729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:03.965729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.965729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:03.965729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:03.965729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:03.965729 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.966128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:03.966128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:03.966128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:03.966128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:03.966128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:03.966128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:03.966128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:03.966128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:03.966128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:03.966128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:03.966128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:03.966128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:03.966128 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:03.966560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.966560 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:03.967331 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:03.967868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:03.968495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:03.968495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.968495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:03.981749 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:03.982146 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:03.982342 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:03.982520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:07:03.982954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:03.983233 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.983954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:03.983954 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:03.983954 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:03.983954 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:03.983954 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:03.983954 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:03.983954 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.983954 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:03.985062 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:03.985585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:03.985585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:03.985585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:03.985585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:03.985585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:03.985585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:03.985585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:03.985585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.985585 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:03.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:03.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:03.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:03.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:03.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:03.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:03.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:07:03.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:03.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:03.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.986404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:03.986756 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:03.987243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:03.987243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:03.987243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.987243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:03.988001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:03.988466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:03.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:03.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:03.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:03.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:03.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:03.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:03.992317 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:03.992539 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:03.992650 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:03.992773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:03.993301 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:03.993666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.994328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:03.994328 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:03.994328 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:03.994328 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:03.994328 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:03.994328 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:03.994328 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.994328 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:03.995031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:03.995446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:03.995446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:03.995446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:03.995446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:03.995446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:03.995446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:03.995446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:03.995446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.995446 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:03.995941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.995941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:03.995941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:03.995941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:03.995941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:03.995941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:03.995941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:03.995941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:03.995941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:03.995941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.995941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:03.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:03.996674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:03.996674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:03.996674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.996674 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:03.997007 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:03.997517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:03.998023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:03.998023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:03.998023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:03.998023 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:04.020783 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:04.021099 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:04.021282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:04.021515 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:04.021903 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:04.022398 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:04.022534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:04.023134 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:04.023134 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:04.023134 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:04.023134 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:04.023134 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:04.023134 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:04.023134 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:04.023134 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:04.024408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:04.024408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:04.024408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:04.024408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:04.024408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:04.024408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:04.024408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:04.024408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:04.024408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:04.024408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:04.024408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:04.024408 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:04.025165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:04.025165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:04.025165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:04.025165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:04.025165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:04.025165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:04.025165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:04.025165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:04.025165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:04.025165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:04.025165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:04.025165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:04.025165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:04.025814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:04.026544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:04.028365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:04.028365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:04.028365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:04.028365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:04.028365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:04.028365 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:06.263611 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:06.264523 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:06.264733 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:06.266198 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:06.304326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:06.304750 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:06.305784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:06.305784 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:06.305784 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:06.305784 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:06.305784 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:06.305784 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:06.305784 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:06.305784 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:06.306690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:06.306690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:06.306690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:06.306690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:06.306690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:06.306690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:06.306690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:06.306690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:06.306690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:06.306690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:06.306690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:06.306690 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:06.307291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:06.307291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:06.307291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:06.307291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:06.307291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:06.307291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:06.307291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:06.307291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:06.307291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:06.307291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:06.307291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:06.307291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:06.307291 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:06.308034 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:06.308726 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:06.309559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:06.309559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:06.309559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:06.309559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:06.309559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:06.309559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:08.961656 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:08.962546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:08.962546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:08.964035 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:08.964551 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:08.964891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:08.965999 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:08.965999 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:08.965999 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:08.965999 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:08.965999 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:08.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.965999 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:08.967194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.967194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:08.967194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:08.967194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:08.967194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:08.967194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:08.967194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:08.967194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.967194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:08.967194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:08.967194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:08.967194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.967872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:08.967872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:08.967872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:08.967872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:08.967872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:08.967872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:08.967872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:08.967872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:08.967872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:08.967872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:08.967872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:08.967872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:08.967872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:08.968538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.968538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:08.968643 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:08.969347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:08.970167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:08.970167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.970167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:08.982450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:08.983454 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:08.983635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:08.983882 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:07:08.984847 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:08.985133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.985919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:08.985919 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:08.985919 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:08.985919 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:08.985919 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:08.985919 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:08.985919 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.985919 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:08.986999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:08.987911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:08.987911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:08.987911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:08.987911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:08.987911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:08.987911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:08.987911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:08.987911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.987911 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:08.988285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.988285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:08.988285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:08.988285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:08.988285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:08.988285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:08.988285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:08.988285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:07:08.988285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:08.988285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:08.988285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.988285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:08.988624 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:08.989008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:08.989008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:08.989008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.989008 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:08.989259 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:08.989686 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:08.990168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:08.990168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:08.990168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:08.990168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:08.990168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:08.990168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.990168 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:08.992735 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:08.993132 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:08.993312 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:08.993657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:08.994264 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:08.994552 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.995226 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:08.995226 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:08.995226 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:08.995226 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:08.995226 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:08.995226 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:08.995226 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.995226 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:08.996146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:08.996583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:08.996583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:08.996583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:08.996583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:08.996583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:08.996583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:08.996583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:08.996583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.996583 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:08.997084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.997084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:08.997084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:08.997084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:08.997084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:08.997084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:08.997084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:08.997084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:08.997084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:08.997084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.997084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:08.997442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:08.997886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:08.997886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:08.997886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.997886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:08.998713 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:08.999165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:08.999688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:08.999688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:08.999688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:08.999688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:09.021359 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:09.021989 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:09.022173 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:09.022521 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:09.023408 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:09.023739 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:09.024004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:09.025020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:09.025020 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:09.025020 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:09.025020 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:09.025020 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:09.025020 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:09.025020 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:09.025020 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:09.026814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:09.026814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:09.026814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:09.026814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:09.026814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:09.026814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:09.026814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:09.026814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:09.026814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:09.026814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:09.026814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:09.026814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:09.027407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:09.027407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:09.027407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:09.027407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:09.027407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:09.027407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:09.027407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:09.027407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:09.027407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:09.027407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:09.027407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:09.027407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:09.027407 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:09.028083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:09.028083 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:09.029640 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:09.030138 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:09.030727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:09.030727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:09.030727 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:11.264407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:11.265280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:11.265280 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:11.266371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:11.303977 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:11.304234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:11.304752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:11.304752 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:11.304752 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:11.304752 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:11.304752 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:11.304752 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:11.304752 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:11.304752 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:11.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:11.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:11.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:11.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:11.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:11.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:11.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:11.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:11.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:11.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:11.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:11.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:11.306002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:11.306002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:11.306002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:11.306002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:11.306002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:11.306002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:11.306002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:11.306002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:11.306002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:11.306002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:11.306002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:11.306002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:11.306002 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:11.306705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:11.307682 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:11.308620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:11.308620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:11.308620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:11.308620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:11.308620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:11.308620 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:13.962314 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:13.962917 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:13.962961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:13.964065 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:13.964410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:13.964622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.965463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:13.965463 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:13.965463 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:13.965463 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:13.965463 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:13.965463 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:13.965463 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.965463 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:13.966601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.966601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:13.966601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:13.966601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:13.966601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:13.966601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:13.966601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:13.966601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.966601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:13.966601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:13.966601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:13.966601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.966919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:13.966919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:13.966919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:13.966919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:13.966919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:13.966919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:13.966919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:13.966919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:13.966919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:13.966919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:13.966919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:13.966919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:13.966919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:13.967232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.967232 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:13.967885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:13.968270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:13.968691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:13.968691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.968691 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:13.983164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:13.983497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:13.983741 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:13.984037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:07:13.984593 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:13.984797 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.985570 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:13.985570 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:13.985570 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:13.985570 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:13.985570 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:13.985570 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:13.985570 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.985570 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:13.986945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:13.987705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:13.987705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:13.987705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:13.987705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:13.987705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:13.987705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:13.987705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:13.987705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.987705 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:13.988538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.988538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:13.988538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:13.988538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:13.988538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:13.988538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:13.988538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:13.988538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:07:13.988538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:13.988538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:13.988538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.988538 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:13.989159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:13.989928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:13.989928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:13.989928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.989928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:13.990117 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:13.990833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:13.991678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:13.991678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:13.991678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:13.991678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:13.991678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:13.991678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.991678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:13.993115 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:13.993479 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:13.993609 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:13.993741 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:13.994054 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:13.994255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.994671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:13.994671 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:13.994671 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:13.994671 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:13.994671 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:13.994671 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:13.994671 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.994671 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:13.995409 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:13.995853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:13.995853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:13.995853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:13.995853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:13.995853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:13.995853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:13.995853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:13.995853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.995853 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:13.996312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.996312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:13.996312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:13.996312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:13.996312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:13.996312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:13.996312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:13.996312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:13.996312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:13.996312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.996312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:13.996822 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:13.997249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:13.997249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:13.997249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.997249 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:13.997510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:13.997977 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:13.998512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:13.998512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:13.998512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:13.998512 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:14.021767 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:14.022153 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:14.022430 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:14.022742 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:14.023195 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:14.023487 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:14.023650 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:14.024371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:14.024371 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:14.024371 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:14.024371 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:14.024371 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:14.024371 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:14.024371 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:14.024371 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:14.026054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:14.026054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:14.026054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:14.026054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:14.026054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:14.026054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:14.026054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:14.026054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:14.026054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:14.026054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:14.026054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:14.026054 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:14.026672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:14.026672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:14.026672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:14.026672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:14.026672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:14.026672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:14.026672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:14.026672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:14.026672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:14.026672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:14.026672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:14.026672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:14.026672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:14.027239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:14.027239 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:14.028088 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:14.028786 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:14.029646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:14.029646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:14.029646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:16.264976 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:16.265855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:16.265855 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:16.266938 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:16.268639 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=257 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 16:07:16.300782 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:16.301182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:16.301994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:16.301994 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:16.301994 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:16.301994 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:16.301994 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:16.301994 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:16.301994 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:16.301994 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:16.302744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:16.302744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:16.302744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:16.302744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:16.302744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:16.302744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:16.302744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:16.302744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:16.302744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:16.302744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:16.302744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:16.302744 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:16.303467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:16.303467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:16.303467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:16.303467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:16.303467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:16.303467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:16.303467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:16.303467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:16.303467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:16.303467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:16.303467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:16.303467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:16.303467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:16.304605 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:16.305432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:16.306244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:16.306244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:16.306244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:16.306244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:16.306244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:16.306244 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:18.962922 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:18.963563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:18.963741 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:18.964885 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:18.965260 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:18.965493 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.966351 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:18.966351 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:18.966351 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:18.966351 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:18.966351 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:18.966351 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:18.966351 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.966351 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:18.967784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.967784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:18.967784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:18.967784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:18.967784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:18.967784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:18.967784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:18.967784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.967784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:18.967784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:18.967784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:18.967784 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.968383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:18.968383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:18.968383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:18.968383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:18.968383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:18.968383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:18.968383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:18.968383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:18.968383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:18.968383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:18.968383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:18.968383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:18.968383 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:18.969060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.969060 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:18.969486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:18.969959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:18.970569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:18.970569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.970569 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:18.983558 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:18.983954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:18.984155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:18.984354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:07:18.985098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:18.985463 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.986330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:18.986330 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:18.986330 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:18.986330 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:18.986330 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:18.986330 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:18.986330 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.986330 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:18.987477 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:18.988371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:18.988371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:18.988371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:18.988371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:18.988371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:18.988371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:18.988371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:18.988371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.988371 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:18.988830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.988830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:18.988830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:18.988830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:18.988830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:18.988830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:18.988830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:18.988830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:07:18.988830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:18.988830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:18.988830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.988830 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:18.989486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:18.990156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:18.990156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:18.990156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.990156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:18.990380 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:18.991240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:18.992076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:18.992076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:18.992076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:18.992076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:18.992076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:18.992076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.992076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:18.993595 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:18.993886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:18.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:18.994230 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:18.994627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:18.994872 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.995378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:18.995378 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:18.995378 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:18.995378 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:18.995378 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:18.995378 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:18.995378 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.995378 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:18.996123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:18.996572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:18.996572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:18.996572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:18.996572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:18.996572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:18.996572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:18.996572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:18.996572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.996572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:18.997078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.997078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:18.997078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:18.997078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:18.997078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:18.997078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:18.997078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:18.997078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:18.997078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:18.997078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.997078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:18.997452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:18.997836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:18.997836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:18.997836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.997836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:18.998195 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:18.998681 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:18.999178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:18.999178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:18.999178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:18.999178 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:19.022229 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:19.022664 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:19.022858 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:19.023058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:19.023759 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:19.024139 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:19.024393 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:19.025341 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:19.025341 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:19.025341 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:19.025341 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:19.025341 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:19.025341 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:19.025341 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:19.025341 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:19.026596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:19.026596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:19.026596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:19.026596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:19.026596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:19.026596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:19.026596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:19.026596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:19.026596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:19.026596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:19.026596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:19.026596 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:19.027459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:19.027459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:19.027459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:19.027459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:19.027459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:19.027459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:19.027459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:19.027459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:19.027459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:19.027459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:19.027459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:19.027459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:19.027459 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:19.028289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:19.029328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:19.030191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:19.030191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:19.030191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:19.030191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:19.030191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:19.030191 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:21.265550 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:21.266089 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:21.266147 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:21.266927 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:21.304098 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:21.304518 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:21.305565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:21.305565 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:21.305565 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:21.305565 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:21.305565 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:21.305565 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:21.305565 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:21.305565 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:21.306617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:21.306617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:21.306617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:21.306617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:21.306617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:21.306617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:21.306617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:21.306617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:21.306617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:21.306617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:21.306617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:21.306617 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:21.307486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:21.307486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:21.307486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:21.307486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:21.307486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:21.307486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:21.307486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:21.307486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:21.307486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:21.307486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:21.307486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:21.307486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:21.307486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:21.308429 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:21.309152 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:21.310115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:21.310115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:21.310115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:21.310115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:21.310115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:21.310115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:23.963665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:23.964438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:23.964649 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:23.965723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:23.965961 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:23.966123 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.966889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:23.966889 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:23.966889 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:23.966889 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:23.966889 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:23.966889 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:23.966889 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.966889 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:23.969078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.969078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:23.969078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:23.969078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:23.969078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:23.969078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:23.969078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:23.969078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.969078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:23.969078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:23.969078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:23.969078 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.969649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:23.969649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:23.969649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:23.969649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:23.969649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:23.969649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:23.969649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:23.969649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:23.969649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:23.969649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:23.969649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:23.969649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:23.969649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:23.970179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.970179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:23.971626 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:23.972517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:23.973314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:23.973314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.973314 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:23.983962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:23.984555 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:23.984640 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:23.984894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:07:23.985226 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:23.986447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.986447 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:23.986447 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:23.986447 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:23.986447 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:23.986447 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:23.986447 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:23.986447 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.986447 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:23.987201 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:23.987601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:23.987601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:23.987601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:23.987601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:23.987601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:23.987601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:23.987601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:23.987601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.987601 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:23.988566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.988566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:23.988566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:23.988566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:23.988566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:23.988566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:23.988566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:23.988566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:07:23.988566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:23.988566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:23.988566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.988566 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:23.988949 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:23.989326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:23.989326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:23.989326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.989326 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:23.990194 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:23.991108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:23.991606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:23.991606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:23.991606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:23.991606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:23.991606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:23.991606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.991606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:23.993936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:23.994164 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:23.994331 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:23.994501 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:23.995458 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:23.995458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.995869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:23.995869 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:23.995869 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:23.995869 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:23.995869 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:23.995869 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:23.995869 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.995869 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:23.996467 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:23.996904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:23.996904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:23.996904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:23.996904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:23.996904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:23.996904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:23.996904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:23.996904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.996904 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:23.997462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.997462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:23.997462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:23.997462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:23.997462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:23.997462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:23.997462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:23.997462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:23.997462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:23.997462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.997462 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:23.997933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:23.998328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:23.998328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:23.998328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.998328 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:23.998450 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:23.998898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:23.999452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:23.999452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:23.999452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:23.999452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:24.022577 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:24.022871 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:24.022987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:24.023174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:24.023820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:24.024315 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:24.024599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:24.026404 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:24.026404 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:24.026404 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:24.026404 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:24.026404 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:24.026404 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:24.026404 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:24.026404 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:24.027359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:24.027359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:24.027359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:24.027359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:24.027359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:24.027359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:24.027359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:24.027359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:24.027359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:24.027359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:24.027359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:24.027359 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:24.027958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:24.027958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:24.027958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:24.027958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:24.027958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:24.027958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:24.027958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:24.027958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:24.027958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:24.027958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:24.027958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:24.027958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:24.027958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:24.028608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:24.029363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:24.030287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:24.030287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:24.030287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:24.030287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:24.030287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:24.030287 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:26.266241 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:26.267155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:26.267281 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:26.268720 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:26.302695 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:26.303092 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:26.303999 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:26.303999 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:26.303999 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:26.303999 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:26.303999 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:26.303999 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:26.303999 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:26.303999 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:26.305163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:26.305163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:26.305163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:26.305163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:26.305163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:26.305163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:26.305163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:26.305163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:26.305163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:26.305163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:26.305163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:26.305163 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:26.305954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:26.305954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:26.305954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:26.305954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:26.305954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:26.305954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:26.305954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:26.305954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:26.305954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:26.305954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:26.305954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:26.305954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:26.305954 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:26.306754 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:26.307627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:26.308777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:26.308777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:26.308777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:26.308777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:26.308777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:26.308777 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:28.964617 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:28.965296 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:28.965296 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:28.966502 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:28.966887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:28.967103 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.967804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:28.967804 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:28.967804 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:28.967804 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:28.967804 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:28.967804 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:28.967804 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.967804 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:28.968814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.968814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:28.968814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:28.968814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:28.968814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:28.968814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:28.968814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:28.968814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.968814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:28.968814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:28.968814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:28.968814 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:28.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:28.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:28.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:28.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:28.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:28.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:28.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:28.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:28.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:28.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:28.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:28.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:28.969555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.969555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:28.970000 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:28.970476 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:28.971001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:28.971001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.971001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:28.984535 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:28.984810 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:28.984957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:28.985075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:07:28.985590 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:28.985990 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.986779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:28.986779 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:28.986779 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:28.986779 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:28.986779 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:28.986779 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:28.986779 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.986779 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:28.987534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:28.988272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:28.988272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:28.988272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:28.988272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:28.988272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:28.988272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:28.988272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:28.988272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.988272 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:28.988671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.988671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:28.988671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:28.988671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:28.988671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:28.988671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:28.988671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:28.988671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:07:28.988671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:28.988671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:28.988671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.988671 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:28.989181 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:28.989759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:28.989759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:28.989759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.989759 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:28.990852 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:28.992298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:28.994133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:28.994133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:28.994133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:28.994133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:28.994133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:28.994133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.994133 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:28.994834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:28.994834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:28.994834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:28.994834 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:28.995090 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:28.995362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.995898 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:28.995898 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:28.995898 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:28.995898 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:28.995898 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:28.995898 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:28.995898 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.995898 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:28.996672 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:28.996991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:28.996991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:28.996991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:28.996991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:28.996991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:28.996991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:28.996991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:28.996991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.996991 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:28.997773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.997773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:28.997773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:28.997773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:28.997773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:28.997773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:28.997773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:28.997773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:28.997773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:28.997773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.997773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:28.998017 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:28.998269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:28.998269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:28.998269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.998269 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:28.999197 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:28.999536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:29.000357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:29.000357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:29.000357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:29.000357 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:29.023074 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:29.023612 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:29.023821 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:29.024093 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:29.024653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:29.025017 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:29.025258 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:29.026095 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:29.026095 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:29.026095 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:29.026095 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:29.026095 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:29.026095 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:29.026095 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:29.026095 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:29.027498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:29.027498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:29.027498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:29.027498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:29.027498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:29.027498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:29.027498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:29.027498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:29.027498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:29.027498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:29.027498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:29.027498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:29.028042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:29.028042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:29.028042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:29.028042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:29.028042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:29.028042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:29.028042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:29.028042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:29.028042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:29.028042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:29.028042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:29.028042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:29.028042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:29.028598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:29.028598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:29.029099 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:29.029836 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:29.030606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:29.030606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:29.030606 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:31.266793 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:31.267608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:31.267608 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:31.268468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:31.300863 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:31.301037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:31.301572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:31.301572 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:31.301572 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:31.301572 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:31.301572 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:31.301572 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:31.301572 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:31.301572 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:31.302370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:31.302370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:31.302370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:31.302370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:31.302370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:31.302370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:31.302370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:31.302370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:31.302370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:31.302370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:31.302370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:31.302370 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:31.302986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:31.302986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:31.302986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:31.302986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:31.302986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:31.302986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:31.302986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:31.302986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:31.302986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:31.302986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:31.302986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:31.302986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:31.302986 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:31.303563 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:31.304241 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:31.305154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:31.305154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:31.305154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:31.305154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:31.305154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:31.305154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:33.965253 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:33.965981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:33.967380 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:33.967580 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:33.967876 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:33.968112 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:33.969159 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:33.969159 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:33.969159 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:33.969159 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:33.969159 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:33.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.969159 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:33.971243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.971243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:33.971243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:33.971243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:33.971243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:33.971243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:33.971243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:33.971243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:33.971243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:33.971243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:33.971243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:33.971243 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:33.971810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:33.971810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:33.971810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:33.971810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:33.971810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:33.971810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:33.971810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:33.971810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:33.971810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:33.971810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:33.971810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:33.971810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:33.971810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:33.972443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.972443 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:33.973400 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:33.974255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:33.975032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:33.975032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.975032 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:33.985007 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:33.985194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:33.985308 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:33.985369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:07:33.985909 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:33.986329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.987288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:33.987288 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:33.987288 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:33.987288 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:33.987288 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:33.987288 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:33.987288 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.987288 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:33.987833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:33.988572 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:33.989283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:33.989283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:33.989283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:07:33.989283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:33.989283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:33.989283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:33.989283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:33.989283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:33.989283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:33.989283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:33.989283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:33.989283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:33.989283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:33.989875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:33.989875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:33.989875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:33.989875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:33.989875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:33.989875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:33.989875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:33.989875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:33.989875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:33.989875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:33.989875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.989875 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:33.990581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:33.991283 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:33.992082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:33.992082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:33.992082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:33.992082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:33.992082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:33.992082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.992082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:33.994905 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:33.995788 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:33.996434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:33.996434 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:33.996627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:33.996794 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.997423 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:33.997423 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:33.997423 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:33.997423 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:33.997423 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:33.997423 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:33.997423 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.997423 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:33.998525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:33.998811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:33.998811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:33.998811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:33.998811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:33.998811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:33.998811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:33.998811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:33.998811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:33.998811 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:34.000030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:34.000030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:34.000030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:34.000030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:34.000030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:34.000030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:34.000030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:34.000030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:34.000030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:34.000030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:34.000030 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:34.000277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:34.000565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:34.000565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:34.000565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:34.000565 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:34.001966 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:34.002306 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:34.002646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:34.002646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:34.002646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:34.002646 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:34.023578 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:34.023867 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:34.024027 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:34.024300 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:34.024864 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:34.025563 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:34.025885 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:34.026894 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:34.026894 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:34.026894 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:34.026894 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:34.026894 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:34.026894 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:34.026894 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:34.026894 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:34.028165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:34.028165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:34.028165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:34.028165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:34.028165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:34.028165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:34.028165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:34.028165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:34.028165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:34.028165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:34.028165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:34.028165 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:34.028764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:34.028764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:34.028764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:34.028764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:34.028764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:34.028764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:34.028764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:34.028764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:34.028764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:34.028764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:34.028764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:34.028764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:34.028764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:34.029164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:34.029164 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:34.029715 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:34.030228 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:34.030680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:34.030680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:34.030680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:36.267545 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:36.268483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:36.268483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:36.269672 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:36.302354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:36.302354 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:36.303127 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:36.303127 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:36.303127 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:36.303127 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:36.303127 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:36.303127 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:36.303127 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:36.303127 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:36.303855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:36.303855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:36.303855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:36.303855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:36.303855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:36.303855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:36.303855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:36.303855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:36.303855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:36.303855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:36.303855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:36.303855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:36.304534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:36.304534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:36.304534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:36.304534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:36.304534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:36.304534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:36.304534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:36.304534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:36.304534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:36.304534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:36.304534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:36.304534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:36.304534 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:36.305188 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:36.306546 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:36.307788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:36.307788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:36.307788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:36.307788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:36.307788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:36.307788 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:38.965947 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:38.966484 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:38.966644 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:38.967614 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:38.967883 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:38.968100 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.968930 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:38.968930 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:38.968930 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:38.968930 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:38.968930 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:38.968930 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:38.968930 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.968930 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:38.970403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.970403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:38.970403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:38.970403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:38.970403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:38.970403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:38.970403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:38.970403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.970403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:38.970403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:38.970403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:38.970403 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.970832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:38.970832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:38.970832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:38.970832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:38.970832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:38.970832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:38.970832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:38.970832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:38.970832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:38.970832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:38.970832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:38.970832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:38.970832 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:38.971204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.971204 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:38.971725 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:38.972329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:38.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:38.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.972881 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:38.985419 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:38.985723 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:38.985894 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:38.986061 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:07:38.986559 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:38.986849 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.987660 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:38.987660 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:38.987660 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:38.987660 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:38.987660 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:38.987660 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:38.987660 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.987660 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:38.988844 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:38.989374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:38.989374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:38.989374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:38.989374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:38.989374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:38.989374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:38.989374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:38.989374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.989374 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:38.990108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.990108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:38.990108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:38.990108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:38.990108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:38.990108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:38.990108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:38.990108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:07:38.990108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:38.990108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:38.990108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.990108 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:38.990608 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:38.991018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:38.991018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:38.991018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.991018 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:38.991543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:38.992031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:38.992554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:38.992554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:38.992554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:38.992554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:38.992554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:38.992554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.992554 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:38.995978 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:38.996292 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:38.996406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:38.996533 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:38.996920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:38.997279 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.997868 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:38.997868 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:38.997868 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:38.997868 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:38.997868 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:38.997868 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:38.997868 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.997868 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:38.998607 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:38.999040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:38.999040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:38.999040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:38.999040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:38.999040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:38.999040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:38.999040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:38.999040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.999040 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:38.999539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:38.999539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:38.999539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:38.999539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:38.999539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:38.999539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:38.999539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:38.999539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:38.999539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:38.999539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.999539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:38.999889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:39.000273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:39.000273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:39.000273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:39.000273 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:39.000653 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:39.001154 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:39.001679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:39.001679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:39.001679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:39.001679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:39.024068 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:39.024404 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:39.024508 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:39.024654 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:39.025021 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:39.025371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:39.025571 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:39.026167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:39.026167 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:39.026167 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:39.026167 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:39.026167 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:39.026167 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:39.026167 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:39.026167 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:39.027005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:39.027005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:39.027005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:39.027005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:39.027005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:39.027005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:39.027005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:39.027005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:39.027005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:39.027005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:39.027005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:39.027005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:39.027458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:39.027458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:39.027458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:39.027458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:39.027458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:39.027458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:39.027458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:39.027458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:39.027458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:39.027458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:39.027458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:39.027458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:39.027458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:39.027816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:39.027816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:39.027963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:39.028461 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:39.029020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:39.029020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:39.029020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:41.268086 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:41.268919 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:41.269058 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:41.270125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:41.307998 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:41.308598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:41.309680 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:41.309680 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:41.309680 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:41.309680 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:41.309680 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:41.309680 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:41.309680 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:41.309680 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:41.310564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:41.310564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:41.310564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:41.310564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:41.310564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:41.310564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:41.310564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:41.310564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:41.310564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:41.310564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:41.310564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:41.310564 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:41.311167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:41.311167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:41.311167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:41.311167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:41.311167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:41.311167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:41.311167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:41.311167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:41.311167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:41.311167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:41.311167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:41.311167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:41.311167 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:41.311837 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:41.312670 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:41.313667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:41.313667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:41.313667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:41.313667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:41.313667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:41.313667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:43.966554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:43.967459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:43.967459 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:43.968282 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:43.968892 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:43.968892 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.969992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:43.969992 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:43.969992 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:43.969992 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:43.969992 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:43.969992 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:43.969992 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.969992 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:43.971958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.971958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:43.971958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:43.971958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:43.971958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:43.971958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:43.971958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:43.971958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.971958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:43.971958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:43.971958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:43.971958 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.972610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:43.972610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:43.972610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:43.972610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:43.972610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:43.972610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:43.972610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:43.972610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:43.972610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:43.972610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:43.972610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:43.972610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:43.972610 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:43.973156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.973156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:43.974411 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:43.975376 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:43.976238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:43.976238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.976238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:43.985763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:43.986075 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:43.986201 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:43.986365 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:07:43.986848 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:43.987124 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.987824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:43.987824 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:43.987824 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:43.987824 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:43.987824 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:43.987824 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:43.987824 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.987824 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:43.988648 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:43.989454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:43.989454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:43.989454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:43.989454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:43.989454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:43.989454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:43.989454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:43.989454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.989454 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:43.989899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.989899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:43.989899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:43.989899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:43.989899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:43.989899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:43.989899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:43.989899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:07:43.989899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:43.989899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:43.989899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.989899 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:43.990497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:43.991077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:43.991747 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:43.992599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:43.992599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:43.992599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:43.992599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:43.992599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:43.992599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:43.992599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:43.992599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:43.992599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:43.992599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:43.992599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.992599 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:43.996438 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:43.996657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:43.996735 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:43.996839 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:43.997352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:43.997724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.998362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:43.998362 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:43.998362 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:43.998362 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:43.998362 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:43.998362 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:43.998362 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.998362 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:43.998922 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:43.999352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:43.999352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:43.999352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:43.999352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:43.999352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:43.999352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:43.999352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:43.999352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.999352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:43.999600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:43.999600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:43.999600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:43.999600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:43.999600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:43.999600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:43.999600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:43.999600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:43.999600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:43.999600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.999600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:43.999945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:44.000329 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:44.000739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:44.001281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:44.001281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:44.001281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:44.001281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:44.001281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:44.001281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:44.001281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:44.001281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:44.001281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:44.001281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:44.001281 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:44.024657 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:44.025096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:44.025318 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:44.025593 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:44.026040 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:44.026395 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:44.026635 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:44.027363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:44.027363 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:44.027363 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:44.027363 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:44.027363 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:44.027363 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:44.027363 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:44.027363 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:44.028878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:44.028878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:44.028878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:44.028878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:44.028878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:44.028878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:44.028878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:44.028878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:44.028878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:44.028878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:44.028878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:44.028878 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:44.029466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:44.029466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:44.029466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:44.029466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:44.029466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:44.029466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:44.029466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:44.029466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:44.029466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:44.029466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:44.029466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:44.029466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:44.029466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:44.030009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:44.030009 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:44.030767 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:44.031479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:44.032295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:44.032295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:44.032295 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:46.268915 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:46.269722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:46.269722 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:46.271006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:46.306538 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:46.307020 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:46.307959 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:46.307959 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:46.307959 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:46.307959 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:46.307959 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:46.307959 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:46.307959 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:46.307959 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:46.308838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:46.308838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:46.308838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:46.308838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:46.308838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:46.308838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:46.308838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:46.308838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:46.308838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:46.308838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:46.308838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:46.308838 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:46.309424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:46.309424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:46.309424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:46.309424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:46.309424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:46.309424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:46.309424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:46.309424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:46.309424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:46.309424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:46.309424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:46.309424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:46.309424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:46.309992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:46.310739 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:46.311701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:46.311701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:46.311701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:46.311701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:46.311701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:46.311701 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:48.967327 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:48.968801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:48.968801 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:48.969371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:48.969735 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:48.969973 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.970843 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:48.970843 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:48.970843 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:48.970843 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:48.970843 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:48.970843 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:48.970843 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.970843 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:48.973389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.973389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:48.973389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:48.973389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:48.973389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:48.973389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:48.973389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:48.973389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:48.973389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:48.973389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:48.973389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:48.973389 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:48.974217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:48.974217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:48.974217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:48.974217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:48.974217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:48.974217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:48.974217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:48.974217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:48.974217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:48.974217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:48.974217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:48.974217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:48.974217 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:48.975157 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:48.977338 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:48.978510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:48.978510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:48.978510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:48.978510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:48.978510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.978510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:48.986291 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:48.986597 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:48.986709 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:48.986829 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:07:48.988042 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:48.988042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.988146 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:48.988146 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:48.988146 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:48.988146 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:48.988146 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:48.988146 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:48.988146 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.988146 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:48.988929 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:48.989466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:48.989466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:48.989466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:48.989466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:48.989466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:48.989466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:48.989466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:48.989466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.989466 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:48.989762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.989762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:48.989762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:48.989762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:48.989762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:48.989762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:48.989762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:48.989762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:07:48.989762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:48.989762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:48.989762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:48.989762 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:48.990322 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:48.990897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:48.991290 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:48.991941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:48.991941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:48.991941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:48.991941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:48.991941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:48.991941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:48.991941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:48.991941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:48.991941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:48.991941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:48.991941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.991941 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:48.996918 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:48.997593 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:48.997804 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:48.998021 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:48.998471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:48.999268 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.999497 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:48.999497 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:48.999497 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:48.999497 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:48.999497 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:48.999497 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:48.999497 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:48.999497 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:49.000910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:49.001503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:49.001503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:49.001503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:49.001503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:49.001503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:49.001503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:49.001503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:49.001503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:49.001503 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:49.002602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:49.002602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:49.002602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:49.002602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:49.002602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:49.002602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:49.002602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:49.002602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:49.002602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:49.002602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:49.002602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:49.003012 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:49.003420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:49.003420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:49.003420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:49.003420 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:49.004140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:49.004664 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:49.005251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:49.005251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:49.005251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:49.005251 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:49.025371 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:49.025857 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:49.026040 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:49.026369 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:49.028349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:49.028349 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:49.028349 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:49.028602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:49.028602 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:49.028602 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:49.028602 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:49.028602 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:49.028602 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:49.028602 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:49.028602 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:49.030156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:49.030156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:49.030156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:49.030156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:49.030156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:49.030156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:49.030156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:49.030156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:49.030156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:49.030156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:49.030156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:49.030156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:49.030576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:49.030576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:49.030576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:49.030576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:49.030576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:49.030576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:49.030576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:49.030576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:49.030576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:49.030576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:49.030576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:49.030576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:49.030576 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:49.031026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:49.031026 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:49.031913 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:49.032377 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:49.033015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:49.033015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:49.033015 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:51.269420 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:51.270009 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:51.270161 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:51.271468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:51.305683 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:51.305683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:51.306010 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:51.306010 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:51.306010 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:51.306010 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:51.306010 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:51.306010 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:51.306010 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:51.306010 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:51.306855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:51.306855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:51.306855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:51.306855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:51.306855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:51.306855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:51.306855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:51.306855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:51.306855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:51.306855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:51.306855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:51.306855 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:51.307506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:51.307506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:51.307506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:51.307506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:51.307506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:51.307506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:51.307506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:51.307506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:51.307506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:51.307506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:51.307506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:51.307506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:51.307506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:51.308145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:51.309027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:51.309995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:51.309995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:51.309995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:51.309995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:51.309995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:51.309995 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:53.968020 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:53.969006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:53.969006 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:53.970296 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:53.970296 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:53.970575 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.971551 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:53.971551 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:53.971551 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:53.971551 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:53.971551 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:53.971551 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:53.971551 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.971551 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:53.972775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.972775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:53.972775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:53.972775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:53.972775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:53.972775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:53.972775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:53.972775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:53.972775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:53.972775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:53.972775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:53.972775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:53.973581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:53.973581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:53.973581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:53.973581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:53.973581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:53.973581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:53.973581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:53.973581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:53.973581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:53.973581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:53.973581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:53.973581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:53.973581 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:53.974210 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:53.974963 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:53.975935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:53.975935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:53.975935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:53.975935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:53.975935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.975935 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:53.988033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:53.988033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:53.988033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:53.988033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:07:53.988033 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:53.988033 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.988297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:53.988297 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:53.988297 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:53.988297 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:53.988297 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:53.988297 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:53.988297 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.988297 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:53.989120 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:53.989517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:53.989517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:53.989517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:53.989517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:53.989517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:53.989517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:53.989517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:53.989517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.989517 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:53.989998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.989998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:53.989998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:53.989998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:53.989998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:53.989998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:53.989998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:53.989998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:07:53.989998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:53.989998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:53.989998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:53.989998 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:53.990310 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:53.990604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:53.990604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:53.990604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.990604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:53.991304 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:53.991828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:53.992401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:53.992401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:53.992401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:53.992401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:53.992401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:53.992401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:53.992401 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:53.997330 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:53.997936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:53.997936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:53.997936 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:53.998410 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:53.998766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:54.002248 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:54.002867 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.003285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:54.003285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:54.003285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:54.003285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:54.003285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:54.003285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:54.003285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:54.003285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:54.003285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:54.003285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:54.003285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:54.003285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:54.003285 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:54.003649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:54.003649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:54.003649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:54.003649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:54.003649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:54.003649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:54.003649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:54.003649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:54.003649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:54.003649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:54.003649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:54.003649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:54.003649 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:54.003976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.003976 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:54.004266 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:54.004765 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:54.005296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:54.005296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:54.005296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.005296 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:54.025838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:54.026778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:54.026778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:54.026778 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:54.027333 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:54.027690 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:54.027919 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:54.031220 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:54.031667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:54.032003 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:54.032384 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:54.032862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:54.032862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:54.032862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:54.032862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:54.032862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:54.032862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:54.032862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:54.032862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:54.032862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:54.032862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:56.270106 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:56.270933 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:56.270933 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:56.271341 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:56.312189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:56.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:56.313645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:56.313645 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:56.313645 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:56.313645 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:56.313645 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:56.313645 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:56.313645 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:56.313645 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:56.314683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:56.314683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:56.314683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:56.314683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:56.314683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:56.314683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:56.314683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:56.314683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:56.314683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:56.314683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:56.314683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:56.314683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:56.315086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:56.315086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:56.315086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:56.315086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:56.315086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:56.315086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:56.315086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:56.315086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:56.315086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:56.315086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:56.315086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:56.315086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:56.315086 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:56.315471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:56.315471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:56.315721 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:56.316196 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:56.316737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:56.316737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:56.316737 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:58.968737 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:58.970758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:58.970758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:58.970943 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:58.971336 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:58.971642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.972591 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:58.972591 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:58.972591 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:58.972591 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:58.972591 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:58.972591 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:58.972591 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.972591 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:58.974267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.974267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:58.974267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:58.974267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:58.974267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:58.974267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:58.974267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:58.974267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:58.974267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:58.974267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:58.974267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:58.974267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:58.975347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:58.975347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:58.975347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:58.975347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:58.975347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:58.975347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:58.975347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:58.975347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:58.975347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:58.975347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:58.975347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:58.975347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:58.975347 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:58.975796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.975796 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:58.976135 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:58.976742 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:58.977398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:58.977398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.977398 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:58.987126 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:58.987553 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:58.987755 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:58.987954 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:07:58.988500 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:58.988820 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.989600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:58.989600 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:58.989600 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:58.989600 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:58.989600 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:58.989600 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:58.989600 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.989600 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:58.990932 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:58.991862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:58.991862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:58.991862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:58.991862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:58.991862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:58.991862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:58.991862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:58.991862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.991862 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:58.992396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.992396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:58.992396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:58.992396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:58.992396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:58.992396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:58.992396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:58.992396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:07:58.992396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:58.992396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:58.992396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:58.992396 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:58.993082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:58.993753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:58.993753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:58.993753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.993753 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:58.994046 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:58.994816 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:58.995679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:58.995679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:58.995679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:58.995679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:58.995679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:58.995679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:58.995679 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:58.997795 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:58.998076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:58.998174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:58.998348 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:58.998925 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:58.999362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.000048 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:59.000048 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:59.000048 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:59.000048 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:59.000048 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:59.000048 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:59.000048 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.000048 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:07:59.000947 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:07:59.001442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:59.001442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:07:59.001442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:59.001442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:07:59.001442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:07:59.001442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:59.001442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:07:59.001442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.001442 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:07:59.001746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.001746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:59.001746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:59.001746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:59.001746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:59.001746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:59.001746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:59.001746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:59.001746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:59.001746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:59.001746 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:59.002097 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:59.002559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:59.002559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:59.002559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.002559 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:59.002683 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:59.003203 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:59.004749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:59.004749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:59.004749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.004749 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:07:59.026411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:07:59.026997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:07:59.027340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:07:59.027698 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:07:59.028184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:59.028673 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:07:59.028880 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.029667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:07:59.029667 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:59.029667 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:07:59.029667 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:07:59.029667 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:59.029667 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:07:59.029667 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.029667 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:07:59.031537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.031537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:07:59.031537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:59.031537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:07:59.031537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:07:59.031537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:59.031537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:59.031537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:59.031537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:59.031537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:59.031537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:07:59.031537 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:59.032140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:07:59.032140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:59.032140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:07:59.032140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:59.032140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:07:59.032140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:07:59.032140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:59.032140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:07:59.032140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:59.032140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:07:59.032140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:07:59.032140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:59.032140 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:07:59.032766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.032766 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:07:59.033573 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:07:59.034352 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:07:59.035301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:07:59.035301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:07:59.035301 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:01.270701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:01.273354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:01.273354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:01.273354 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:01.311946 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:01.312183 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:01.312826 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:01.312826 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:01.312826 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:01.312826 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:01.312826 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:01.312826 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:01.312826 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:01.312826 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:01.313779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:01.313779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:01.313779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:01.313779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:01.313779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:01.313779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:08:01.313779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:01.313779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:01.313779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:01.313779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:01.313779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:01.313779 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:01.314657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:01.314657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:01.314657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:01.314657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:01.314657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:01.314657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:01.314657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:01.314657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:01.314657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:01.314657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:01.314657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:01.314657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:01.314657 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:01.315334 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:01.316153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:01.317080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:01.317080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:01.317080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:01.317080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:01.317080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:01.317080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:03.969482 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:03.970064 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:03.970166 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:03.971273 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:03.971637 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:03.971910 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.972982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:03.972982 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:03.972982 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:03.972982 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:03.972982 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:03.972982 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:03.972982 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.972982 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:03.974050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.974050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:03.974050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:03.974050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:03.974050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:03.974050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:08:03.974050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:03.974050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:03.974050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:03.974050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:03.974050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:03.974050 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:03.974373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:03.974373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:03.974373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:03.974373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:03.974373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:03.974373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:03.974373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:03.974373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:03.974373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:03.974373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:03.974373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:03.974373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:03.974373 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:03.974719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.974719 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:03.975298 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:03.976077 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:03.976526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:03.976526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.976526 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:03.987680 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:03.988184 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:03.988333 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:03.988463 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:08:03.989124 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:03.989452 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.990234 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:03.990234 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:03.990234 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:03.990234 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:03.990234 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:03.990234 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:03.990234 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.990234 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:08:03.991031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:03.992027 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:03.992602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:08:03.992602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:08:03.992602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:08:03.992602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:03.992602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:03.992602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:03.992602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:03.992602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:03.992602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:03.992602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:03.992602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:03.992602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:03.992602 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:03.993176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:03.993176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:03.993176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:03.993176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:03.993176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:03.993176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:03.993176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:03.993176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:03.993176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:03.993176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:03.993176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.993176 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:03.993665 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:03.995126 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:03.996722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:03.996722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:03.996722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:03.996722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:03.996722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:03.996722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:03.996722 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:03.998387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:03.998828 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:03.998957 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:03.999076 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:03.999534 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:03.999773 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.000378 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:04.000378 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:04.000378 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:04.000378 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:04.000378 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:04.000378 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:04.000378 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.000378 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:08:04.001190 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:08:04.001945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:04.001945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:08:04.001945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:04.001945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:08:04.001945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:08:04.001945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:04.001945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:08:04.001945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.001945 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:08:04.002479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.002479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:04.002479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:04.002479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:04.002479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:04.002479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:08:04.002479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:08:04.002479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:04.002479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:04.002479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:04.002479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:04.003115 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:04.003803 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:04.004541 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:04.005422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:04.005422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:04.005422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:04.005422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:04.005422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:04.005422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:04.005422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:04.005422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:04.005422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:04.005422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.005422 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:04.026900 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:04.027507 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:04.027636 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:04.027819 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:04.028468 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:04.028835 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:04.029052 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.030001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:04.030001 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:04.030001 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:04.030001 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:04.030001 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:04.030001 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:04.030001 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.030001 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:04.031024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.031024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:04.031024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:04.031024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:04.031024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:04.031024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:08:04.031024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:04.031024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:04.031024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:04.031024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:04.031024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:04.031024 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:04.031611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:04.031611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:04.031611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:04.031611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:04.031611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:04.031611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:04.031611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:04.031611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:04.031611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:04.031611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:04.031611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:04.031611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:04.031611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:04.032305 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:04.033085 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:04.033979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:04.033979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:04.033979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:04.033979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:04.033979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:04.033979 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:06.272066 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:06.272873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:06.272873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:06.274049 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:06.310335 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:06.310723 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:06.311600 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:06.311600 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:06.311600 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:06.311600 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:06.311600 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:06.311600 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:06.311600 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:06.311600 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:06.312362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:06.312362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:06.312362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:06.312362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:06.312362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:06.312362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:08:06.312362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:06.312362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:06.312362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:06.312362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:06.312362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:06.312362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:06.312971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:06.312971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:06.312971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:06.312971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:06.312971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:06.312971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:06.312971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:06.312971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:06.312971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:06.312971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:06.312971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:06.312971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:06.312971 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:06.313539 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:06.314302 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:06.315069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:06.315069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:06.315069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:06.315069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:06.315069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:06.315069 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:08.970189 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:08.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:08.971194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:08.971997 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:08.972352 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:08.972604 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.973469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:08.973469 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:08.973469 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:08.973469 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:08.973469 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:08.973469 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:08.973469 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.973469 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:08.974957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.974957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:08.974957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:08.974957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:08.974957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:08.974957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:08:08.974957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:08.974957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:08.974957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:08.974957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:08.974957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:08.974957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:08.975782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:08.975782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:08.975782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:08.975782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:08.975782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:08.975782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:08.975782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:08.975782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:08.975782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:08.975782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:08.975782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:08.975782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:08.975782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:08.976536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.976536 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:08.976724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:08.977645 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:08.978689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:08.978689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.978689 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:08.988116 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:08.988511 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:08.988687 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:08.988872 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:08:08.989387 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:08.989590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.990177 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:08.990177 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:08.990177 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:08.990177 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:08.990177 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:08.990177 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:08.990177 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.990177 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:08:08.991402 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:08:08.992330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:08.992330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:08:08.992330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:08.992330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:08:08.992330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:08:08.992330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:08.992330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:08:08.992330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.992330 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:08:08.992856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.992856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:08.992856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:08.992856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:08.992856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:08.992856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:08:08.992856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:08:08.992856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:08:08.992856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:08.992856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:08.992856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:08.992856 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:08.993528 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:08.994143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:08.994143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:08.994143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.994143 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:08.994491 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:08.995255 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:08.996082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:08.996082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:08.996082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:08.996082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:08.996082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:08.996082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:08.996082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:08.998797 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:08.999045 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:08.999155 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:08.999302 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:08.999635 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:08.999891 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.000413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:09.000413 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:09.000413 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:09.000413 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:09.000413 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:09.000413 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:09.000413 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.000413 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:08:09.001199 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:08:09.001675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:09.001675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:08:09.001675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:09.001675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:08:09.001675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:08:09.001675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:09.001675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:08:09.001675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.001675 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:08:09.002067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.002067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:09.002067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:09.002067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:09.002067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:09.002067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:08:09.002067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:08:09.002067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:09.002067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:09.002067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:09.002067 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:09.002498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:09.003016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:09.003016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:09.003016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.003016 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:09.003145 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:09.003752 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:09.004368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:09.004368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:09.004368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.004368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:09.027483 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:09.027849 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:09.028225 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:09.028520 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:09.028987 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:09.029407 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:09.029535 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.030180 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:09.030180 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:09.030180 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:09.030180 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:09.030180 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:09.030180 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:09.030180 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.030180 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:09.031630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.031630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:09.031630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:09.031630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:09.031630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:09.031630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:08:09.031630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:09.031630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:09.031630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:09.031630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:09.031630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:09.031630 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:09.032368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:09.032368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:09.032368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:09.032368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:09.032368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:09.032368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:09.032368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:09.032368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:09.032368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:09.032368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:09.032368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:09.032368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:09.032368 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:09.032920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.032920 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:09.033362 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:09.034031 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:09.034931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:09.034931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:09.034931 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:11.272820 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:11.276318 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:11.276318 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:11.276318 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:11.313544 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:11.313544 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:11.313792 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:11.313792 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:11.313792 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:11.313792 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:11.313792 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:11.313792 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:11.313792 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:11.313792 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:11.314824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:11.314824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:11.314824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:11.314824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:11.314824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:11.314824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:08:11.314824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:11.314824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:11.314824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:11.314824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:11.314824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:11.314824 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:11.315498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:11.315498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:11.315498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:11.315498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:11.315498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:11.315498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:11.315498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:11.315498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:11.315498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:11.315498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:11.315498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:11.315498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:11.315498 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:11.316042 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:11.316775 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:11.317555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:11.317555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:11.317555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:11.317555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:11.317555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:11.317555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:13.970852 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:13.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:13.971773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:13.972554 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:13.972728 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:13.972895 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.973612 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:13.973612 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:13.973612 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:13.973612 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:13.973612 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:13.973612 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:13.973612 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.973612 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:13.974964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.974964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:13.974964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:13.974964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:13.974964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:13.974964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:08:13.974964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:13.974964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:13.974964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:13.974964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:13.974964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:13.974964 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:13.975915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:13.975915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:13.975915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:13.975915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:13.975915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:13.975915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:13.975915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:13.975915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:13.975915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:13.975915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:13.975915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:13.975915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:13.975915 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:13.976893 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:13.978028 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:13.979588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:13.979588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:13.979588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:13.979588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:13.979588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.979588 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:13.988450 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:13.988945 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:13.989136 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:13.989406 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:08:13.989939 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:13.990288 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.991096 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:13.991096 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:13.991096 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:13.991096 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:13.991096 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:13.991096 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:13.991096 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.991096 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:08:13.992458 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:08:13.993545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:13.993545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:08:13.993545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:13.993545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:08:13.993545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:08:13.993545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:13.993545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:08:13.993545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.993545 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:08:13.994179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.994179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:13.994179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:13.994179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:13.994179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:13.994179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:08:13.994179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:08:13.994179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:08:13.994179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:13.994179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:13.994179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:13.994179 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:13.995005 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:13.995928 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:13.996869 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:13.998299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:13.998299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:13.998299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:13.998299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:13.998299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:13.998299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:13.998299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:13.998299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:13.998299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:13.998299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:13.998299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:13.998299 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:13.999154 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:13.999646 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:13.999837 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:14.000078 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:14.000528 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:14.000782 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.001324 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:14.001324 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:14.001324 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:14.001324 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:14.001324 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:14.001324 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:14.001324 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.001324 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:08:14.002084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:08:14.002698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:14.002698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:08:14.002698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:14.002698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:08:14.002698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:08:14.002698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:14.002698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:08:14.002698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.002698 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:08:14.003090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.003090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:14.003090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:14.003090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:14.003090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:14.003090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:08:14.003090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:08:14.003090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:14.003090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:14.003090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:14.003090 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:14.003641 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:14.004432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:14.004432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:14.004432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.004432 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:14.004627 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:14.005297 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:14.006037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:14.006037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:14.006037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.006037 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:14.027970 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:14.028594 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:14.028766 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:14.029060 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:14.029907 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:14.030411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:14.030699 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.031652 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:14.031652 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:14.031652 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:14.031652 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:14.031652 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:14.031652 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:14.031652 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.031652 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:14.032908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.032908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:14.032908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:14.032908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:14.032908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:14.032908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:08:14.032908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:14.032908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:14.032908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:14.032908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:14.032908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:14.032908 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:14.033381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:14.033381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:14.033381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:14.033381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:14.033381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:14.033381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:14.033381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:14.033381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:14.033381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:14.033381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:14.033381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:14.033381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:14.033381 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:14.033810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.033810 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:14.033981 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:14.034471 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:14.035035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:14.035035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:14.035035 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:16.274653 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:16.275491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:16.275491 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:16.276660 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:16.278472 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG dbcounter [-] [105503] Writing DB stats neutron:SELECT=240 {{(pid=105503) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 07 16:08:16.307013 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:16.307845 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:16.308481 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:16.308481 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:16.308481 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:16.308481 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:16.308481 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:16.308481 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:16.308481 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:16.308481 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:16.309021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:16.309021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:16.309021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:16.309021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:16.309021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:16.309021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:08:16.309021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:16.309021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:16.309021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:16.309021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:16.309021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:16.309021 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:16.309688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:16.309688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:16.309688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:16.309688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:16.309688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:16.309688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:16.309688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:16.309688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:16.309688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:16.309688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:16.309688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:16.309688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:16.309688 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:16.310289 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:16.310951 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:16.312070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:16.312070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:16.312070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:16.312070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:16.312070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:16.312070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:18.971955 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:18.972963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:18.972963 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:18.973764 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:18.974037 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:18.974238 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.975044 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:18.975044 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:18.975044 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:18.975044 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:18.975044 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:18.975044 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:18.975044 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.975044 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:18.976080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.976080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:18.976080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:18.976080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:18.976080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:18.976080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:08:18.976080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:18.976080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:18.976080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:18.976080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:18.976080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:18.976080 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:18.976851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:18.976851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:18.976851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:18.976851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:18.976851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:18.976851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:18.976851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:18.976851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:18.976851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:18.976851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:18.976851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:18.976851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:18.976851 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:18.977598 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:18.978622 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:18.979889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:18.979889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:18.979889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:18.979889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:18.979889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.979889 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:18.989125 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:18.989674 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:18.989873 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:18.990070 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:08:18.990559 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:18.990985 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.991469 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:18.991469 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:18.991469 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:18.991469 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:18.991469 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:18.991469 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:18.991469 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.991469 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:08:18.993070 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:08:18.993903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:18.993903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:08:18.993903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:18.993903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:08:18.993903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:08:18.993903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:18.993903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:08:18.993903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.993903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:08:18.994828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.994828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:18.994828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:18.994828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:18.994828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:18.994828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:08:18.994828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:08:18.994828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:08:18.994828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:18.994828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:18.994828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:18.994828 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:18.995413 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:18.995992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:18.995992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:18.995992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.995992 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:18.996486 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:18.997267 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:18.998082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:18.998082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:18.998082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:18.998082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:18.998082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:18.998082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:18.998082 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:18.999576 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:18.999887 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:19.000003 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:19.000133 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:19.000546 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:19.000804 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.001271 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:19.001271 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:19.001271 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:19.001271 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:19.001271 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:19.001271 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:19.001271 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.001271 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:08:19.002001 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:08:19.002482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:19.002482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:08:19.002482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:19.002482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:08:19.002482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:08:19.002482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:19.002482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:08:19.002482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.002482 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:08:19.003277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.003277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:19.003277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:19.003277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:19.003277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:19.003277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:08:19.003277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:08:19.003277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:19.003277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:19.003277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:19.003277 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:19.004073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:19.004741 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:19.005505 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:19.006076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:19.006076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:19.006076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:19.006076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:19.006076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:19.006076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:19.006076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:19.006076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:19.006076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:19.006076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.006076 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:19.028763 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:19.029227 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:19.029340 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:19.029497 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:19.029758 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:19.029981 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:19.030121 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.030631 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:19.030631 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:19.030631 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:19.030631 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:19.030631 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:19.030631 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:19.030631 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.030631 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:19.031562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.031562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:19.031562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:19.031562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:19.031562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:19.031562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:08:19.031562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:19.031562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:19.031562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:19.031562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:19.031562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:19.031562 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:19.032182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:19.032182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:19.032182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:19.032182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:19.032182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:19.032182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:19.032182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:19.032182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:19.032182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:19.032182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:19.032182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:19.032182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:19.032182 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:19.032764 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:19.033510 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:19.034343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:19.034343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:19.034343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:19.034343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:19.034343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:19.034343 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:21.275435 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:21.278838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:21.278838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:21.278838 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:21.309891 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:21.310375 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:21.311004 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:21.311004 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:21.311004 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:21.311004 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:21.311004 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:21.311004 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:21.311004 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:21.311004 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:21.312284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:21.312284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:21.312284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:21.312284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:21.312284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:21.312284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:08:21.312284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:21.312284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:21.312284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:21.312284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:21.312284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:21.312284 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:21.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:21.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:21.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:21.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:21.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:21.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:21.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:21.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:21.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:21.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:21.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:21.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:21.312716 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:21.313074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:21.313074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:21.314355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:21.314707 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:21.315156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:21.315156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:21.315156 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:23.972734 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:23.976270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:23.976270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:23.976270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:23.976270 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:23.976270 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.976903 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:23.976903 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:23.976903 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:23.976903 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:23.976903 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:23.976903 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:23.976903 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.976903 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:23.979051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.979051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:23.979051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:23.979051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:23.979051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:23.979051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:08:23.979051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:23.979051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:23.979051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:23.979051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:23.979051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:23.979051 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:23.979982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:23.979982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:23.979982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:23.979982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:23.979982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:23.979982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:23.979982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:23.979982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:23.979982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:23.979982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:23.979982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:23.979982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:23.979982 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:23.980807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.980807 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:23.983906 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:23.984614 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:23.985153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:23.985153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.985153 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:23.989701 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:23.991262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:23.991262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:23.991262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:08:23.991262 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:23.991262 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.991632 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:23.991632 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:23.991632 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:23.991632 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:23.991632 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:23.991632 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:23.991632 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.991632 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:08:23.994224 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:23.994687 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:23.995173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:08:23.995173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:08:23.995173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:08:23.995173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:23.995173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:23.995173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:23.995173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:23.995173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:23.995173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:23.995173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:23.995173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:23.995173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:23.995173 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:23.995666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:23.995666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:23.995666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:23.995666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:23.995666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:23.995666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:23.995666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:23.995666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:23.995666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:23.995666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:23.995666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.995666 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:23.997355 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:23.998084 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:23.998887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:23.998887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:23.998887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:23.998887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:23.998887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:23.998887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:23.998887 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:24.000114 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:24.002073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:24.002073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:24.002073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:24.002073 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:24.002073 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.002808 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:24.002808 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:24.002808 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:24.002808 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:24.002808 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:24.002808 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:24.002808 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.002808 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:08:24.004424 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:08:24.005087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:24.005087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:08:24.005087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:24.005087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:08:24.005087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:08:24.005087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:24.005087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:08:24.005087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.005087 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:08:24.008093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.008093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:24.008093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:24.008093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:24.008093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:24.008093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:08:24.008093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:08:24.008093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:24.008093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:24.008093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:24.008093 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:24.008506 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:24.008923 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:24.009363 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:24.009897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:24.009897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:24.009897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:24.009897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:24.009897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:24.009897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:24.009897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:24.009897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:24.009897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:24.009897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.009897 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:24.029357 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:24.030326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:24.030326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:24.030326 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:24.030808 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:24.031194 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:24.031957 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.032274 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:24.032274 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:24.032274 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:24.032274 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:24.032274 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:24.032274 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:24.032274 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.032274 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:24.035240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.035240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:24.035240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:24.035240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:24.035240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:24.035240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:08:24.035240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:24.035240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:24.035240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:24.035240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:24.035240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:24.035240 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:24.036025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:24.036025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:24.036025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:24.036025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:24.036025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:24.036025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:24.036025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:24.036025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:24.036025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:24.036025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:24.036025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:24.036025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:24.036025 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:24.036886 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:24.037864 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:24.038667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:24.038667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:24.038667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:24.038667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:24.038667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:24.038667 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:26.276671 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:26.277471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:26.277471 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:26.278627 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:26.312771 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:26.312909 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:26.313312 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:26.313312 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:26.313312 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:26.313312 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:26.313312 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:26.313312 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:26.313312 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:26.313312 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:26.313950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:26.313950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:26.313950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:26.313950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:26.313950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:26.313950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:08:26.313950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:26.313950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:26.313950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:26.313950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:26.313950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:26.313950 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:26.314397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:26.314397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:26.314397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:26.314397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:26.314397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:26.314397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:26.314397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:26.314397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:26.314397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:26.314397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:26.314397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:26.314397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:26.314397 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:26.315041 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:26.315543 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:26.316317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:26.316317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:26.316317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:26.316317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:26.316317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:26.316317 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:28.974174 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:28.974920 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:28.975023 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:28.976141 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-165b5214-095f-4c34-9f9c-9f0a42407cb7, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:28.976458 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-0938bfb8-1052-49e2-bd69-e5fda45e8a2f, routes=[], if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:28.976685 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.977525 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:28.977525 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:28.977525 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:28.977525 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:28.977525 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:28.977525 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:28.977525 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.977525 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:28.978912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.978912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:28.978912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:28.978912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:28.978912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:28.978912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:08:28.978912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:28.978912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:28.978912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:28.978912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:28.978912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:28.978912 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:28.979448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:28.979448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:28.979448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:28.979448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:28.979448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:28.979448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:28.979448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:28.979448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:28.979448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:28.979448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:28.979448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:28.979448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:28.979448 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:28.980006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.980006 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:28.981771 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:28.982523 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:28.983074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:28.983074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.983074 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:28.990198 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:28.990579 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:28.990777 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:28.990962 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting fip-distributed flag in NB_Global to False {{(pid=105503) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 07 16:08:28.991411 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:28.991833 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.992555 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:28.992555 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:28.992555 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:28.992555 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:28.992555 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:28.992555 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:28.992555 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.992555 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:08:28.993731 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:08:28.994386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:28.994386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:08:28.994386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:28.994386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:08:28.994386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:08:28.994386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:28.994386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:08:28.994386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.994386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:08:28.995479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.995479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:28.995479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:28.995479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:28.995479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:28.995479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:08:28.995479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:08:28.995479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 07 16:08:28.995479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:28.995479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:28.995479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:28.995479 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:28.996049 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:28.996642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:28.996642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:28.996642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.996642 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:28.997063 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                                                           ^^^^^^^^
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:28.997815 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:28.998611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:28.998611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:28.998611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:28.998611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:28.998611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:28.998611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:28.998611 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:29.000489 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:29.000710 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_ha_failover {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:29.000773 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:29.000886 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:29.001096 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:29.001286 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.001740 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:29.001740 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:29.001740 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:29.001740 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:29.001740 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:29.001740 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:29.001740 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.001740 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 07 16:08:29.002229 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 07 16:08:29.002669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:29.002669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 07 16:08:29.002669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:29.002669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 07 16:08:29.002669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 07 16:08:29.002669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:29.002669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 07 16:08:29.002669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.002669 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.command 
May 07 16:08:29.003253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.003253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:29.003253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:29.003253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:29.003253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:29.003253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:08:29.003253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:08:29.003253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:29.003253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:29.003253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:29.003253 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:29.003590 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:29.003994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:29.003994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:29.003994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.003994 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:29.004137 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:29.004933 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:29.005495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:29.005495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:29.005495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.005495 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics 
May 07 16:08:29.030029 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=105503) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 07 16:08:29.030377 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 07 16:08:29.030495 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=105503) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 07 16:08:29.030665 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=105503) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 07 16:08:29.030980 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:29.031188 np0000172501 neutron-ovn-maintenance-worker[105503]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=105503) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 07 16:08:29.031386 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.031817 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Traceback (most recent call last):
May 07 16:08:29.031817 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:29.031817 np0000172501 neutron-ovn-maintenance-worker[105503]:     txn.results.put(txn.do_commit())
May 07 16:08:29.031817 np0000172501 neutron-ovn-maintenance-worker[105503]:                     ^^^^^^^^^^^^^^^
May 07 16:08:29.031817 np0000172501 neutron-ovn-maintenance-worker[105503]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:29.031817 np0000172501 neutron-ovn-maintenance-worker[105503]:     raise RuntimeError(msg)
May 07 16:08:29.031817 np0000172501 neutron-ovn-maintenance-worker[105503]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.031817 np0000172501 neutron-ovn-maintenance-worker[105503]: 
May 07 16:08:29.032613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.032613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 07 16:08:29.032613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:29.032613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 07 16:08:29.032613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 07 16:08:29.032613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:08:29.032613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:29.032613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:29.032613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:29.032613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:29.032613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 07 16:08:29.032613 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:29.032978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 07 16:08:29.032978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:29.032978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 07 16:08:29.032978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:29.032978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 07 16:08:29.032978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 07 16:08:29.032978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:29.032978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 07 16:08:29.032978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:29.032978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 07 16:08:29.032978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 07 16:08:29.032978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:29.032978 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 07 16:08:29.033385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.033385 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics [None req-8ba2bc2f-7c7e-4c32-a6fe-49a50783db3e None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics Traceback (most recent call last):
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     work()
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(*args, **kwargs)
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 07 16:08:29.033678 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     next(self.gen)
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     with self.create_transaction(
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     self.result = self.commit()
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise result.ex
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 07 16:08:29.034171 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 07 16:08:29.034724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics     raise RuntimeError(msg)
May 07 16:08:29.034724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 07 16:08:29.034724 np0000172501 neutron-ovn-maintenance-worker[105503]: ERROR futurist.periodics